View Issue Details

IDProjectCategoryView StatusLast Update
0004787MMW v4DB/FileMonitorpublic2009-05-09 12:24
Reporterpeke Assigned To 
PriorityimmediateSeverityminorReproducibilityalways
Status closedResolutionfixed 
Product Version3.1 
Fixed in Version3.1 
Summary0004787: Network Database Access Optimization.
DescriptionI have set intensive Add/Rescan to database on two PCs on Network where Database was on third one. I get lots of SQL Errors All stated "unable to open database file (14, 14)" in each case if I press Retry MM continues and process is finished without error.

After debugging and excluding possible issues the problem is more likely with Exclusive Access to Whole Library instead of just Record (like it was done on Access DB)

--

Users have also reported numerous performance problems with using MM 3.x over the network.
Additional Informationhttp://www.mediamonkey.com/support/staff/index.php?_m=tickets&_a=viewticket&ticketid=1056
http://www.mediamonkey.com/support/staff/index.php?_m=tickets&_a=viewticket&ticketid=2413&offset=20#

http://www.mediamonkey.com/support/staff/index.php?_m=tickets&_a=viewticket&ticketid=2539&highlight=--+Mixed+--&ticketsearchid=973
This ticket also refers to numerous forum reports:
http://www.mediamonkey.com/forum/viewtopic.php?f=1&t=22023&p=160006&hilit=lan#p160006
http://www.mediamonkey.com/forum/viewtopic.php?f=1&t=31590&p=161744&hilit=lan#p161744
http://www.mediamonkey.com/forum/viewtopic.php?f=7&t=31998&p=162909&hilit=lan#p162909
http://www.mediamonkey.com/forum/viewtopic.php?f=1&t=24816&p=169566&hilit=lan#p169566
http://www.mediamonkey.com/forum/viewtopic.php?f=7&t=30708&p=170582&hilit=nas#p170582
TagsNo tags attached.
Fixed in build1243

Relationships

has duplicate 0004781 resolvedjiri Multiple instances of MM can trigger Error dialog. 

Activities

peke

2008-07-15 16:37

developer   ~0014371

Please check if there is way to Optimize Exclusive Access to library.

Use your Laptop as second PC and just set .INI value DBName=\\<Server>\MM.DB on both PCs and start Add To Library of two different folderers/paths with Large number of files.

rusty

2008-07-18 05:57

administrator   ~0014390

Raising to 'urgent'.

petr

2008-08-20 21:28

developer   ~0014457

SQLite doesn't support record locking (http://www.sqlite.org/faq.html#q5).

peke

2008-11-04 00:14

developer   ~0014854

Rising to immediate for evaluation and Richard Donovan was very helpful to narrow the issue.

I'm Able to replicate it each time and also can confirm that Retry Always Fixes The issue and MM continue to function without any corruption.

Maybe some sort of Exclusion list would be useful or some Retry count per thread should solve that issue.

peke

2008-11-04 00:14

developer   ~0014855

Reminder sent to: jiri, rusty

jiri

2008-11-04 08:58

administrator   ~0014857

SQLite isn't overly optimized for concurent access from several machines over network (that mentioned table locking is one of the reasons).

In order to improve this we can:
1. In case error #14 is returned, retry the action several times (but at most for several seconds).
2. A real fix would be to support some real client-server SQL solution in addition to the default SQLite. However, this isn't a trivial fix and so I don't see it too urgent. On the other hand, local home networks are becoming more common and so some good support for this could be an interesting new feature. That said, it would require also improvements in other areas, e.g. support for several users (different Rating and Playcounts for all of them, etc.).

peke

2008-11-04 14:29

developer   ~0014858

1. That should be more than enough, in most cases error only hapen when Analyze Volume is in progress on some of PCs and until next track is being analyzed.

2. In most cases like in my network only one PC is allowed to Change/Update library the others only access Library to read tracks and be able to play where only Play Counter, Last Played is updated, maybe temporal solution would be that other PCs on network would access in restricted mode similar to Party Mode where users would be only be able to access library but not be able to change it and/or make exclude for that error only and make some sort of Delayed write where MM will make priority management and internal communication where delayed process would be notified on finish. Second thing would be that users uses own PCs to Sync with their Devices and have common/shared library.

This is based on all Tickets raised, user feedback and personal testing.

Just an thought: I wonder if some User specific settings can be read from local HDD in Separate library and Shared Library will contain info on Files and would be more restrictive to access. Restriction can be automatically Applied if DBname= Contains UNC path to Library and PC that have full Access would have Some sort Exception.

Note: Issue is easily replicated on single PC if Library is on NAS where LAN (WAN) Bandwidth slows MM Write to library.

rusty

2008-11-04 23:17

administrator   ~0014863

I actually think that this issue is quite a significant regression. MM is supposed to be a more advanced music manager, and have encouraged users to use it in networked environments (even iTunes and WMP have some capabilities of sharing their libraries over a local network).

The fact that it doesn't work correctly, is an oversight on our part--users who upgraded from MM 2.x to MM 3.x certainly didn't expect to face the troubles they're seeing.

So we have 2 options:
1) Change our positioning--get rid of any mention of using MM in a networked environment (where the DB is shared over a network), and update our faq to indicate that this is no longer supported.

2) Make some sort of fix that would allow MM 3.x to work at least as well in a networked environment as MM 2.x did.

fyi: I've read that Amarok switched from sqlite to mysql so that it can either be used with full mysql or embedded mysql.

I'm hoping that 2) is a possibility for 3.1.

rusty

2008-11-07 15:20

administrator   ~0014874

Per offline discussion:

The network performance problems are most likely related to having more than a single client connected to the DB. See:
http://www.mail-archive.com/sqlite-users@sqlite.org/msg34066.html

Jiri has examined some of the solutions such as those described here:
http://www.sqlite.org/cvstrac/wiki?p=SqliteNetwork
http://modsqlite.sourceforge.net/
However, they would require a significant amount of work, and would not necessarily solve the problem.

Thus the only real long-term solution would be to migrate to a Client/Server solution, however, it's unlikely that this could be achieved in the 3.1 timeframe.

In the short term though, we can minimize errors; whenever SQLite throws any internal error (not an SQL errors), MM would try to repeatedly call last sql command in some timeout (like 1s) for some time (5 calls) and then show up an error message if sqlite still throws errors.

petr

2008-11-30 23:05

developer   ~0015266

I've added code to repeat query when any internal error code will appear. Resolving for 3.1. Will be in 1197.

petr

2009-05-01 20:33

developer   ~0017654

I've optimized code for better networking (speed and errors handling). Will be in 1243.

peke

2009-05-09 11:40

developer   ~0017762

Verified 1244