View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0007478 | MMW v4 | Main Panel | public | 2011-03-07 19:14 | 2011-07-11 02:53 |
| Reporter | petr | Assigned To | |||
| Priority | urgent | Severity | major | Reproducibility | always |
| Status | closed | Resolution | fixed | ||
| Product Version | 4.0 | ||||
| Target Version | 4.0 | Fixed in Version | 4.0 | ||
| Summary | 0007478: Performance issues: scanning is very slow on large DB | ||||
| Description | With large DB (+100t tracks), scanning is very slow and laggy. | ||||
| Tags | No tags attached. | ||||
| Fixed in build | 1354 | ||||
|
|
Made some improvements in 1352 (about 1/3 faster scan). |
|
|
There is still some space for improvement. Steps to reproduce (note use Artist that have larger number of songs like Elvis or Beatles): 1. Add one song of desired artist to Now Playing 2. Copy Artist name to clipboard 3. Right click find more from same -> Artist 4. Click on Search Icon and search only Artist name (you copied in step 2) it is slower than (step 3 and query should be same) 5. Using Back/Next Tree view icons compare showing speed Other than that speed improved is obvious in 1352 like petr noted at least 1/3 faster. If risky i would leave further optimization for MM 4.1 |
|
|
Here is a second set of scanning-related performance issues that were discovered in 1352: 1) MM4 vs MM3 showed some performance decrease, though it may be attributable to the fact that MM3 was a standard build, vs MM4 was a debug build. I notice, though, that during scans with MM4, the scanning process seems to stall periodically unlike with MM3. 2) Within MM4, there was a noticeable performance degradation when: a) scan occurred while a node was selected that required the view to be updated. Some performance degradation was observed in AA view, but was MUCH more severe when the column browser was disabled (likely related to issue 0007494) b) the number of tracks being scanned grows (i.e. track 10 is scanned ~2-3 times as fast as track 6000) in the case of the visible nodes. Test Results (on clean DB) ------------------------------- Non-visible nodes: means scan was performed while focus rested on the Now Playing node Visible nodes: means that scan was performed while focus rested on the Music/Location/Drive/.../Music/All node, in AA View with the Column Browser disabled MM3: non-visible node: 3m30s MM3: visible node (Locations/All): 20m3s MM4: non-visible node: 7m:18s MM4: visible node: 9:17 (AA/Column Browser), 25m+ (AA/NO Column browser) Test with non-clean DB containing 0001256:0004000 tracks (from other directories) ------------------------------- MM4: non-visible node: 10m26s MM4: visible node (Locations/All): 30m30s (debug log posted to ftp server) |
|
|
My results: MM3: non-visible node: 42s MM4: non-visible node: 28s |
|
|
I've tried on slower PC with almost 7k music tracks: MM3: non-visible node: 1m 20s MM3: visible node:2m 5s MM4: non-visible node: 45s MM4: visible node: 1m 25s All tests are done in Details view without Column browser. After some improvements: MM4: visible node with column browser: 54s MM4: visible node with column browser and AA view: 1m 54s (this definitely need to be optimized) --> after some optimizations now it took 55s |
|
|
fyi, comment from a user re. the size of the thumbnails causing slowdowns: http://www.mediamonkey.com/forum/viewtopic.php?f=6&t=55511 |
|
|
Tested build 1353 with the same test as in #7478c23641. Scan with focus on: Now Playing: 2:12 Location/Music/All (AA view): 15:01 Music node (AA view): 6:56 Music node (Details view): 4:15 Performance is now faster than MM3. Nice! That said, it seems that Location node for some reason has performance issues. All other nodes have a performance penalty as well, but not as severe. |
|
|
As promised Petr over IM to repeat tests using 1352 with 1353 and non Debug build. I can also came to conclusion that 1353 is now faster than MM3 when used with same scan settings. Just want to say that we also noticed that MM scanning is mostly using single CPU 70:30 in 2 core system and 70:10:10:10 in 4 core system. I'm guessing that we can improve that also Dividing Scan folders to different thread and Reading AA and generating Thumbnails to different cores/threads. One additional note that using DBGView to create Scanning Debug slows scanning more than 80%. I've rescanned more than 40k of tracks and speed is ~5% faster than MM3. |
|
|
To summarize: - Thumbnail size is now tracked in 0007515. - I don't think that scanning using >1 threads would help much, since it's mostly about disk speed. - The only that Petr should yet review is that Location node slowdown. Feel free to resolve in case there isn't anything obvious. |
|
|
Optimized a location node a little, but i'm not sure we can do anything more there. Fixed in 1354. |
|
|
2. It could improve as it will leave main scanning thread free, also most of our users have multiple HDDs, NAS, LAN where HDD speed is not in question. Also it will improve to fill DB calls in one thread and file reading in other where DB thread will be self closing and would be called when reading thread is done. 3. Verified Location Node in 1358 it is noticeably faster |
|
|
Setting as resolved, it doesn't seem that anything else would really help in real use. |
|
|
Reminder sent to: jiri, petr, rusty Re verified in 1364 due the some user complains in forum. Small note: Disabling Tools-> Options -> -> library -> Generate Thumbnails for Videos increases performance noticeably Maybe we should consider making it disabled by default for MM 4.1? |
|
|
I think the downside of having videos appear without thumbnails is much worse than a 1-time performance penalty associated with thumbnail generation. |
|
|
I agree but we should note that somewhere KB Article or like for Duplicate tracks add "(Takes extra time)"? Especially as all our tests show MM4 faster than MM3. |
|
|
No additional change required. |
|
|
Due the some test from Ludek with MM3 I retested this with 1403 and Closing. |
