View Issue Details

IDProjectCategoryView StatusLast Update
0019156MMW 5Extensions frameworkpublic2022-08-31 20:14
Reporterzvezdan Assigned To 
PriorityurgentSeverityminorReproducibilityN/A
Status closedResolutionreopened 
Target Version5.0.4Fixed in Version5.0.4 
Summary0019156: Changes made with executeQueryAsync are not reflected in the program
DescriptionChanges made to the database by executeQueryAsync method are reflected in the tracklists only after restart of the program. It doesn't help to change nodes or to do anything else in the program, including Optimize database and Rebuild full-text search index.

I have several add-ons for old program that are doing massive changes to the database using SQL and these changes are reflected in the program immediately after SDB.MainTracksWindow.Refresh, which is important for their normal use.

EDIT: This is more serious than I though. The changes made with executeQueryAsync method are unreflected not only in tracklists, but anywhere else in the program, including the Properties dialog and JavaScript code.
Steps To Reproduce- Start the program with an empty database;
- add one test media file into the database;
- open DevTools and type:
app.db.executeQueryAsync("UPDATE Songs SET SongTitle = 'BlahBlah' WHERE ID = 1").then(function() {var oTracklist = app.db.getTracklist("SELECT * FROM Songs WHERE ID = 1", -1); oTracklist.whenLoaded().then(function() {oTracklist.locked(function () {console.log('Title: ' + oTracklist.getValue(0).title)});});});
-> you will get the old Title displayed in DevTools and in the program, e.g. in the tracklist or in the Properties dialog;
- open the database into SQLiteSpy or any other SQLite program and select the Songs table -> SongTitle is "BlahBlah"; or
- restart the program and Title of the file is "BlahBlah".

It is not only the case with DevTools to blame it, you could execute the same code in add-on and you will get the same problem.
TagsNo tags attached.
Fixed in build2652

Relationships

related to 0019157 closedLudek Problems with deleteFolderAsync method 

Activities

Ludek

2022-06-08 18:38

developer   ~0068451

Last edited: 2022-06-08 18:46

Yes, I can confirm that the values are cached, so if there is already a track instance in memory cached then this instance is used (instead of reading from DB again). I believe that this used to be the case in MM4 too, but probably not so noticable as in MM5 the unused tracklists are cached e.g. for minute (just in case a user returns to the previous view in a while) and probably also the garbage collector might not free it immediatelly.

So in your usecase above you should rather use something like this:
var list = app.db.getTracklist("SELECT * FROM Songs WHERE ID = 1", -1);
await list .whenLoaded();
listForEach( list , (itm) => { itm.title = 'BlahBlah'; });
list.commitAsync(); // saves the metadata to the DB and file tag

This way you are sure that the new title is written to the file tag also + reflected everwhere in the UI where this track instance(s) are used.

Alternativelly I could introduce a new method like app.db.clearSharedObjectsCache() that would force re-reading all the metadata from the DB (having performance penalty of course) that could be used by scripters to clear the memory cache (after modifing DB). The cons would be possible data loss for uncommited data. But ok, let's suppose that it is scripters responsibility / user risk.

zvezdan

2022-06-08 19:50

updater   ~0068452

As I said, I have add-ons that are doing massive changes on the database:
1. It is not about one or two or ten or even a hundred tracks, but maybe a whole library, so executeQueryAsync is a way faster than getTracklist(), than itm.title = 'BlahBlah';, than commitAsync();
2. I am doing changes not only on Songs table, but on other tables as well, so the getTracklist() is unusable.
3. I am using the combination of several executeQueryAsync and getTracklist methods that are all interconnected and directly dependent. Restart of the program after each such command is out of question. It would be the same as if you require from users to restart the program after each change in the Properties dialog.
4. Changes made by executeQueryAsync are not reflected even to deleteFolderAsync, which could lead to the unwanted removal of media files and/or records from the Songs table (I mentioned these problems in (https://www.ventismedia.com/mantis/view.php?id=19157).

So, it would be really nice if you add the mentioned method for clearing the cache. I am not sure about what "possible data loss for uncommited data" you are talking about. Aren't all data committed to db before use of executeQueryAsync?

Ludek

2022-06-08 21:38

developer   ~0068454

OK, I can add the method for clearing the cache.

As for uncommited data: I mean e.g. cases when user edits Properties of a track and did not press [OK] button of the Properties dialog yet. But OK, this is mostly probably unrelated to your scripts as I suppose that user does not perform edits when running your scripts? in addition edits that do not need confirmation are commited as soons as possible.

zvezdan

2022-06-08 21:57

updater   ~0068455

No, my scripts are not running when user has opened Properties or any other modal dialog box. At least not the one that is almost finished and waiting for your update. But if they are running in background, I could warn users that they need to stop any editing.

One question about suggested new method. Should I call it after each executeQueryAsync and before each getTracklist or deleteFolderAsync or any other method that read data from cache? Or it will be sufficient to clear cache on the start of series of these commands?

As I said, I have many such methods executed in series, both executeQueryAsync and getTracklist intertwined, and it would be most practical if I could use such clear method just once on the start of such series.

zvezdan

2022-06-08 22:16

updater   ~0068456

I am just wondering, could you do something with the beginTransaction/commitTransaction, instead of adding a new method?

For example, the cache could be cleared on beginTransaction, and after that the other commands including getTracklist will not read data from the cache but directly from the database, until commitTransaction when you could save database to the cache again.

Ludek

2022-06-14 11:17

developer   ~0068522

I'll introduce the clearDataCache method into next build, example:

// navigate to Music > All tracks
await app.db.executeQueryAsync("UPDATE Songs SET SongTitle = 'BlahBlah' WHERE ID = 1");
await app.db.clearDataCache();
uitools.refreshView(); // to reload the tracklist

Ludek

2022-06-14 19:33

developer   ~0068527

Finally calling it just clearCache, added in 5.0.4.2652
image.png (29,496 bytes)   
image.png (29,496 bytes)   

zvezdan

2022-06-14 19:37

updater   ~0068528

Thanks!

zvezdan

2022-06-21 09:17

updater   ~0068641

Just to mention few issues (trying to update drives/paths by SQL):
1. uitools.refreshView() after app.db.clearCache() works very rarely when called from dialog box;
2. it doesn't update paths of files previously placed to Playing list when drive is changed by SQL; I am using app.player.getSongList().notifyChanged(), but it updates paths only if it is applied after selecting a node with the updated drive. I have implemented a workaround for that in my script, but it would be nice if you resolve this somehow in the program.

zvezdan

2022-07-02 14:07

updater   ~0068770

Open the program with empty database, add one test track and select Entire Library / All tracks node. Here is the same example code:
        await app.db.executeQueryAsync("UPDATE Songs SET SongTitle = 'BlahBlah' WHERE ID = 1");
        await app.db.clearCache();
        uitools.refreshView();

When executed from the dialog box, the track in the main tracklist is not updated.

However, there is still even more serious issue with deleteFolderAsync that I already mentioned:
1. open the program with empty database and add one test track with the following path c:\Music\Test.mp3;
2. select Entire Library / All tracks node and make sure that the program shows that path;
3. a) move that file to c:\Temp\Test.mp3 using any external program, e.g. Total Commander; or
3. b) execute this:
        await app.filesystem.renameFileAsync('c:\\Music\\Test.mp3', 'c:\\Temp\\Test.mp3', false /* no overwrite prompt */);
4. execute this from the main window:
        await app.db.executeQueryAsync("UPDATE Songs SET SongPath = ':\\Temp\\Test.mp3' WHERE ID = 1");
        await app.db.clearCache();
        uitools.refreshView();
        await app.filesystem.deleteFolderAsync('c:\\Music\\', false /* no recycle bin */, true /* only if empty */);

and you will get removed the record about that track from the Songs table, even when its path is updated having a new folder. The log file attached.
deleteFolderAsync.log (7,721 bytes)   
00000001	0.00000000	[8832] MM5 [10392](R) **** focusControl: MainToolbar	
00000002	0.13683777	[8832] MM5 [10392](R) BQ: Currently running 1 threads of 10	
00000003	0.13804199	[8832] MM5 [10104](R) BQ: Thread[10104] is executing new task now: Callstack: Script: file:///UpdateLocationOfFiles/init.js ; Func: eval ; Row: 3 ; Col: 26 	
00000004	0.13839620	[8832] MM5 [10104](R) RenameFile: Moving j:\Music\Rock\05 - Shock the Monkey.ogg -> j:\Temp\05 - Shock the Monkey.ogg	
00000005	0.14114861	[8832] MM5 [10104](R) PrepareForTagging: called	
00000006	0.14162995	[8832] MM5 [10104](R) RestoreFromTagging starting.	
00000007	0.14170253	[8832] MM5 [10104](R) RestoreFromTagging finished.	
00000008	0.14178476	[8832] MM5 [10104](R) RenameFile: Success	
00000009	0.14475556	[8832] MM5 [10104](R) BQ: Thread[10104] is executing new task now: Callstack: Script: file:///UpdateLocationOfFiles/init.js ; Func: eval ; Row: 4 ; Col: 18 	
00000010	0.14679983	[8832] MM5 [10104](R) DB exec SQL: UPDATE Songs SET SongPath = ':\Temp\Test.mp3' WHERE ID = 1	
00000011	0.32960069	[8832] MM5 [10104](R) DB lock took 172 ms : UPDATE Songs SET SongPath = ':\Temp\Test.mp3' WHERE ID = 1	
00000012	0.33109167	[8832] MM5 [10104](R) BQ: Thread[10104] is executing new task now: Callstack: Script: file:///UpdateLocationOfFiles/init.js ; Func: eval ; Row: 5 ; Col: 18 	
00000013	0.33476156	[8832] MM5 [10104](R) BQ: Thread[10104] is executing new task now: Callstack: Script: file:///viewHandlers.js ; Func: onHide ; Row: 289 ; Col: 96 	
00000014	0.33476156	[8832] Script: file:///viewHandlers.js ; Func: onHide ; Row: 326 ; Col: 56 	
00000015	0.33476156	[8832] Script: file:///controls/multiview.js ; Func: _process ; Row: 1175 ; Col: 38 	
00000016	0.33476156	[8832] Script: file:///controls/multiview.js ; Func: _handler_call_onHide ; Row: 1183 ; Col: 13 	
00000017	0.33476156	[8832] Script: file:///controls/multiview.js ; Func: reload ; Row: 1194 ; Col: 14 	
00000018	0.33893627	[8832] MM5 [10104](R) BQ: Thread[10104] is executing new task now: Callstack: Script: file:///viewHandlers.js ; Func: __refreshTracklist ; Row: 196 ; Col: 57 	
00000019	0.33893627	[8832] Script: file:///viewHandlers.js ; Func: onShow ; Row: 238 ; Col: 13 	
00000020	0.33893627	[8832] Script: file:///viewHandlers.js ; Func: onShow ; Row: 323 ; Col: 56 	
00000021	0.33893627	[8832] Script: file:///controls/multiview.js ; Func: _process ; Row: 1134 ; Col: 38 	
00000022	0.33893627	[8832] Script: file:///controls/multiview.js ; Func: _handler_call_onShow ; Row: 1140 ; Col: 13 	
00000023	0.33967483	[8832] MM5 [10104](R) DB open SQL: SELECT Songs.*  FROM Songs  	
00000024	0.34000313	[8832] MM5 [10104](R) ***** StandardSongQueryCopy starting 	
00000025	0.34029078	[8832] MM5 [10104](R) ***** StandardSongQueryCopy main loop finishing...  was terminated: false	
00000026	0.34037030	[8832] MM5 [10104](R) ***** StandardSongQueryCopy adding 2	
00000027	0.34049863	[8832] MM5 [10104](R) ***** total tracklist loading time 0ms for 2	
00000028	0.34935677	[8832] MM5 [10104](R) BQ: Thread[10104] is executing new task now: Callstack: Script: file:///UpdateLocationOfFiles/init.js ; Func: eval ; Row: 7 ; Col: 26 	
00000029	0.34945226	[8832] MM5 [10104](R) hasFolderUsefulContent: FALSE, path: j:\Music\Rock\	
00000030	0.34947938	[8832] MM5 [10104](R) Removing folder: j:\Music\Rock\	
00000031	0.34949896	[8832] MM5 [10392](R) BQ: Currently running 2 threads of 10	
00000032	0.35032606	[8832] MM5 [10104](R) PrepareFolderDeletion: PlayerLock locked	
00000033	0.35071793	[8832] MM5 [10104](R) PrepareFolderDeletion: PlayerLock unlocked	
00000034	0.35138419	[8832] MM5 [9892](R) BQ: Thread[9892] is executing new task now: Callstack: Script: file:///controls/listview.js ; Func: eval ; Row: 3494 ; Col: 37 	
00000035	0.35138419	[8832] Script: file:///promise.js ; Func: invokeResolver ; Row: 387 ; Col: 18 	
00000036	0.35138419	[8832] Script: file:///promise.js ; Func: Promise ; Row: 374 ; Col: 7 	
00000037	0.35138419	[8832] Script: file:///controls/listview.js ; Func: setDataSourceSameView ; Row: 3493 ; Col: 38 	
00000038	0.35138419	[8832] Script: file:///viewHandlers.js ; Func: _assign ; Row: 207 ; Col: 50 	
00000039	0.37834632	[8832] MM5 [9892](R) BQ: Thread[9892] is executing new task now: Callstack: Script: file:///controls/trackListView.js ; Func: handle_datasourcechanged ; Row: 2081 ; Col: 20 	
00000040	0.37834632	[8832] Script: file:///controls/listview.js ; Func: set ; Row: 3733 ; Col: 28 	
00000041	0.37834632	[8832] Script: file:///controls/listview.js ; Func: eval ; Row: 3498 ; Col: 37 	
00000042	0.40875486	[8832] MM5 [9892](R) BQ: Thread[9892] is executing new task now: Callstack: Script: file:///controls/control.js ; Func: eval ; Row: 1207 ; Col: 29 	
00000043	0.40875486	[8832] Script: file:///controls/control.js ; Func: eval ; Row: 282 ; Col: 13 	
00000044	0.44843748	[8832] MM5 [9892](R) BQ: Thread[9892] is executing new task now: TSharedList<T>.sortAsync	
00000045	0.44890949	[8832] MM5 [10104](R) Folder - removed	
00000046	0.45639023	[8832] MM5 [10536](R) BQ: Thread[10536] is executing new task now: TSharedBase.RunAsPromiseNative	
00000047	0.45854956	[8832] MM5 [10104](R) TMediaMonkeyApp.notifyCommonChange, type:folder	
00000048	0.45951343	[8832] MM5 [10104](R) 10104 DB prepare SQL: SELECT ID,TrackCount FROM Folders WHERE IDMedia=? AND IDParentFolder=? AND Folder=?	
00000049	0.46072549	[8832] MM5 [10104](R) 10104 DB query prepare finished, took 0.	
00000050	0.46237761	[8832] MM5 [10104](R) DB exec SQL: DELETE FROM Songs WHERE Songs.IDFolder in (SELECT IDChildFolder FROM FoldersHier WHERE IDFolder=8) AND ((1))	
00000051	0.46346164	[8832] MM5 [10536](R) BQ: Thread[10536] is executing new task now: TSharedBase.RunAsPromiseNative	
00000052	0.57965177	[8832] MM5 [10104](R) DB lock took 110 ms : DELETE FROM Songs WHERE Songs.IDFolder in (SELECT IDChildFolder FROM FoldersHier WHERE IDFolder=8) AND ((1))	
00000053	0.57981652	[8832] MM5 [10104](R) DB exec SQL: DELETE FROM Folders WHERE Folders.ID=8	
00000054	0.58008760	[8832] MM5 [10104](R) DB lock took 0 ms : DELETE FROM Folders WHERE Folders.ID=8	
00000055	0.58015293	[8832] MM5 [10104](R) 10104 DB prepare SQL: SELECT ID FROM Albums WHERE Tracks=0	
00000056	0.58031589	[8832] MM5 [10104](R) 10104 DB query prepare finished, took 0.	
00000057	0.58046108	[8832] MM5 [10104](R) 10104 DB prepare SQL: SELECT ID FROM Artists WHERE Tracks=0 AND Albums=0 AND Authors=0 AND Conducts=0 AND Lyrics=0 AND Products=0 AND Roles=0 AND Publishes=0	
00000058	0.58069694	[8832] MM5 [10104](R) 10104 DB query prepare finished, took 15.	
00000059	0.58085418	[8832] MM5 [10104](R) TMediaMonkeyApp.notifyCommonChange, type:dbfolder	
00000060	3.40985489	[8832] MM5 [9892](R) Going 'Worker'to make final callback 9892	
00000061	3.46335888	[8832] MM5 [10536](R) Going 'Worker'to make final callback 10536	
00000062	3.58087707	[8832] MM5 [10104](R) Going 'Worker'to make final callback 10104	
00000063	3.58094549	[8832] MM5 [10104](R) DB: Deleting prepared query: SELECT ID,TrackCount FROM Folders WHERE IDMedia=? AND IDParentFolder=? AND Folder=?	
00000064	3.58100557	[8832] MM5 [10104](R) DB: Deleting prepared query: SELECT ID FROM Albums WHERE Tracks=0	
00000065	3.58102655	[8832] MM5 [10104](R) DB: Deleting prepared query: SELECT ID FROM Artists WHERE Tracks=0 AND Albums=0 AND Authors=0 AND Conducts=0 AND Lyrics=0 AND Products=0 AND Roles=0 AND Publishes=0	
00000066	6.83601236	[8832] MM5 [11084](R) Still waiting for event (callstacks) :	
00000067	6.83604670	[8832] MM5 [11084](R) Wait for event ID :1, delay :60000, callstack :	
00000068	6.83610725	[8832] MM5 [11084](R) Wait for event ID :1, delay :60000, callstack :	
deleteFolderAsync.log (7,721 bytes)   

zvezdan

2022-07-02 14:15

updater   ~0068771

I though that this new clearCache method will prevent deleteFolderAsync from removing records from the Songs table, but it is not. I have another workaround for this, but I am sick with all these workarounds. Especially when sometimes they don't work and users blame me for your quirks.

Ludek

2022-07-14 15:19

developer   ~0068826

Last edited: 2022-07-14 15:27

Re the refreshView in dialogs.
Yes, refreshView is called in the window context, so use code like this to update it in the mainWindow (from which the dialog was called):

uitools.openDialog('dlgPodcastSubscription', {
    modal: true,
    podcast: podcast
}, function (dlg) {
    uitools.refreshView();
});

Re the issue with app.filesystem.deleteFolderAsync : You forgot to update Songs.IDFolder --> this is the reason why the song entry was deleted.

EDIT: Using renameFiles will workarond it: https://www.mediamonkey.com/docs/api/classes/Filesystem.html#method_renameFiles
as it takes Tracklist as param (taking care of all the SQL updates automatically).

Tip: Move discussions like this to the addons dev forum where others can also join the discussion: https://www.mediamonkey.com/forum/viewforum.php?f=27

zvezdan

2022-07-14 17:22

updater   ~0068828

Last edited: 2022-08-31 20:12

Of course I am using refreshView as in your example, but there are situations when you want to update the main window from dialog box, e.g. if it is modeless.

I don't want to use renameFiles because it cannot move non-media files, which I have already reported (0019103), and that doesn't resolve a requisite to update paths after moving files outside of the program.

And I don't want to mess with the Songs.IDFolder and Folders table. They are automatically updated by the program after my workaround. I suppose it is caused by update_songs_songpath trigger.

peke

2022-08-31 20:14

developer   ~0069128

clearCache verified 2661

Closing as like @ludek said in 0019156:0068826 rest of talk should be in devs forum.