View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0006320 | MMW v4 | Framework: Scripts/Extensions | public | 2010-04-03 15:46 | 2022-04-29 01:33 |
| Reporter | peke | Assigned To | |||
| Priority | high | Severity | feature | Reproducibility | N/A |
| Status | closed | Resolution | fixed | ||
| Target Version | 4.0 | Fixed in Version | 4.0 | ||
| Summary | 0006320: Framework: Retrieve URL data | ||||
| Description | In MM4 we have added new download manager and MediaMonkey identify itself as an app that access internet and retrieve data. Currently there is no way to retrieve data from URL using scripting. Adding this will enhance MediaMonkey functionality and enable Script developers to use MM UI for creating custom Panels and Dialogs based on information retrieved from url. By adding few Scripting functions we will create possibility, so that Script developers can create server side applications and retrieve custom data from their sites or parse HTML pages directly in MM for viewing. | ||||
| Tags | No tags attached. | ||||
| Fixed in build | 1307 | ||||
|
|
To make things future proof I suggest to add new methodfunction CreateWebSystem under SDBTools: Function CreateWebSystem As SDBWebSystem |
|
|
CoClass SDBWebSystem/ISDBWebSystem members would be: AssignURL = Property Get/Let AssignURL As String ExecuteURL = Method that Creates background thread to retrieve Data From URL RawData = Property Get RawData As String (Raw stream returned by ExecuteURL) SizeData = Property Get SizeData as Integer (Total Number of Bytes if known) LogInfo = Property Get LogInfo as Integer (HTML error code returned by last ExecuteURL Default 0 or -1) Timeout = Property Get/Let Timeout As String (in Seconds default 30) Events: OnProgress(WebSystem As SDBWebSystem) = Event that will fire on each downloaded part (not sure how indy handle that but in TIE is aprox 4KB) this can be used for creating progress/status bars OnError(WebSystem As SDBWebSystem) = Triggered in case of completion or in case of any error where WebSystem.LogInfo will contain error code |
|
|
Assigned to Rusty to evaluate priority as my thought is that it should be included in first MM 4.0 public release and hammer any possible issues. NOTE: this will prepare MediaMonkey for upcomming addons site and ability to show addon categories within MM tree along with browsing addons thru MediaMonkey |
|
|
Reminder sent to: jiri, Ludek, rusty This is sent due the fact that development time needs to be evaluated. |
|
|
I think that something like DownloadFile( URL as String, FileName as String, ShowProgress as Boolean) would be enough. i.e. scripter could use the function to download data to a custom file and then read it from the file. |
|
|
You are right that would be enough for now. I only wonder if that could be made in separate thread so that downloading would not lock MM until file is finished. I would suggest that we add event that will be fired apron file download finishes. example could lock MM further script execution: '------- DownloadFile("http://download.microsoft.com/download/1/E/5/1E5F1C0A-0D5B-426A-A603-1798B951DDAE/VS2010Express1.iso", "d:\VS2010Express1.iso",True) 'this line will not be executed until DL is done and SDB.ProcessMessages 'would not be executed to allow other background script to work which will eventually lead to MM Freeze '------- My approach would would allow something like this: '------- Dim DLfile Set DLfile = SDB.Tools.WebSystem.CreateNew Script.RegisterEvent DLfile, "OnError", "ErrorHandler" DLfile.AssignURL = "http://download.microsoft.com/download/1/E/5/1E5F1C0A-0D5B-426A-A603-1798B951DDAE/VS2010Express1.iso" DLfile.ExecuteURL Sub ErrorHandler(obj) 'Handling of error/completion of DL '... Script.UnRegisterEvents DLfile End Sub '------- Let me know what you think. |
|
|
I thought that main purposes for scripters should be to have access to the integrated downloader (that is new in MM 4.0). That way I would add new object ISDBDownloader and several new methods like: Dim Downloader as ISDBDownloader Downloader = ISDBApplication::Downloader Downloader.DownloadFile( URL as String, FileName as String, AddToLibrary as Boolean) Downloader.GetDownloadsCount as Integer Downloader.GetDownloadProgress( URL as String) as Integer Downloader.PauseDownload( URL as String) Downloader.ResumeDownload( URL as String) Downloader.CancelDownload( URL as String) That way scripter could easily initiate download of a new audio/video file into library that will be shown under 'Downloads' node in MM UI and can check its progress. I understand that it could be enhanced for the purposes to download a file that wouldn't be shown under Downloads node or a short file that is supposed to be downloaded immediatelly. For purposes like this we can just add additional 2 input parameters to achive this: Downloader.DownloadFile( URL as String, FileName as String, AddToLibrary as Boolean, ShowInUI as Boolean, WaitForDownload as Boolean) if ShowInUI = false then it won't be shown under Downloads node if WaitForDownload = true then it won't be downloaded on the background Nevertheless I am not sure how valuable for scripters these new functions would be, probably nothing at a very high priority. |
|
|
Basically what I want to get with this is to allow scripters to make scripts like this http://www.mediamonkey.com/forum/viewtopic.php?f=2&t=47918&start=0 and retrieve info from internet specifically formatted for MM usage. will bring up small PHP script along with MM VBScript that will show you exactly what teh end result would be. (Note that it will only work MM4.0 and up) |
|
|
Both PHP and VBS is uploaded to FTP. I also created small App that will show PHP behavior under access from different browsers. Example is made emulate MediaMonkey accessing URL Using DownloadFileEX( URL as String, ShowProgress as Boolean) as String; but DownloadFile( URL as String, FileName as String, ShowProgress as Boolean); can be used also. NOTE: Return Data Can be used to create Custom Menus/Forms/Tree Entries in MM On other side Web Developers can add simple code that will format Web page differently in case it is accessed by MM and/or limit accessing to online content only within MM and open possibility to make scripts that interacts with some online resource/site/url. |
|
|
Peke, Re: the topic you posted ( http://www.mediamonkey.com/forum/viewtopic.php?f=2&t=47918&start=0 ) is long and I don't know what exactly you are referring to. As I know the Lyricator script uses its own internet access to download the lyrics. Re: the posted example scripts If I understand correctly you are reffering to that MediaMonkey sets user-agent: MediaMonkey in the HTTP request header and server can format the data based on the info. Similar way could the scripts set the request header or to use the MM API functions like DownloadFile as you pointed. So do you agree with API I suggested in the note 0020173? What do you mean by DownloadFileEX( URL as String, ShowProgress as Boolean) as String; the function is missing destination filename (where to download the file) so how this should work? Should it download the file to the default download location specified in Options? |
|
|
1. Re lyricator: Script was important not the whole post as like you noticed it uses own internet access to download the lyrics which can cause UAC and IE setting issues. Direct result download using MM API should solve those issues. 2. Re example scripts: Yes, as you solved 0004976 the point is where using MM API functions like DownloadFile will tell servers how to respond to HTTP requests both POST and GET and give Scripters ability to make their own versions we talked about in 0003360 NoteID: 18515, 18531 and 18518 For 18518 affiliate sites are directly noted about access from MM due the 0004976 and can make smaller changes if needed to return results back to MM like we talked about in 18515 and XML server results according to common API. 3. I agree with you with proposal in 0020173 but I think it is not currently needed due the right observation of how much of that features is actually needed. 4. re DownloadFileEX: It does not save file at all it returns RAW DATA result received from parsed URL parameter. REGRESSION WARNING: Due the design nature that both DownloadFileEX and DownloadFile blocks scripts execution until function is done MM should execute http://www.mediamonkey.com/wiki/index.php/ISDBApplication::ProcessMessages so the other scripts and MM threads/background processes are not locked. |
|
|
Ok, but I would rather call it "GetURLContent" instead of "DownloadFileEX". And I would add OnDownloadFinished() event that would hit once the download (previously initialized by Downloader.DownloadFile) finishes. So the new API would look like this: Dim Downloader as ISDBDownloader Downloader = ISDBApplication::Downloader Downloader.DownloadFile( URL as String, FileName as String, AddToLibrary as Boolean) Downloader.OnDownloadFinished( URL as String) Downloader.GetURLContent( URL as String) as String Downloader.GetDownloadProgress( URL as String) as Integer Downloader.PauseDownload( URL as String) Downloader.ResumeDownload( URL as String) Downloader.CancelDownload( URL as String) |
|
|
I agree on this as best choice, but to cover all aspects I suggest this change: Downloader.GetDownloadProgress( URL as String, ResultType as Integer) 'Where Second argument should tell if result is an: 0 = Bytes Downloaded (1234567 Bytes), 1 = Percentage (12 %) Additionally to let scripter control all using script two more functions would be needed: Downloader.Status( URL as String) 'Return -1 = Unknown URL, 0 = Finished, 1 = Downloading, 2 = Paused Downloader.URLSize( URL as String) 'Return Estimated URL Stream Size in Bytes reported/returned by server where -1 = Unknown size |
|
|
Script example: ------ Dim Downloader Downloader = SDB.Tools.Downloader 'download file example Downloader.Downloadfile("http://www.happymonkeying.com/beta/mmurltst.php","c:\mmurltest.txt",True) 'download url content Set URLData = Downloader.GetURLContent("http://www.happymonkeying.com/beta/mmurltst.php") 'Retrieve Progress info for URL Set URLProgress = Downloader.GetDownloadProgress("http://www.happymonkeying.com/beta/mmurltst.php",1) 'Second argument Should tell if result is an: 0 = Bytes Downloaded (1234567 Bytes) Default, 1 = Percentage (12 %) 'Pause Download of URL Downloader.PauseDownload("http://www.happymonkeying.com/beta/mmurltst.php") 'Resume URL Download Downloader.ResumeDownload("http://www.happymonkeying.com/beta/mmurltst.php") 'Cancel URL Download Downloader.CancelDownload("http://www.happymonkeying.com/beta/mmurltst.php") 'URL Downloader Status Set URLStatus = Downloader.Status("http://www.happymonkeying.com/beta/mmurltst.php") '-1 = Unknown URL, 0 = Finished, 1 = Downloading, 2 = Paused 'URL Downloader URL Size Set URLSize = Downloader.URLSize("http://www.happymonkeying.com/beta/mmurltst.php") 'Estimated URL Stream Size reported by server (-1 = Unknown size) Sub OnDownloadFinished(arg) 'Downloader.OnDownloadFinished( URL as String) Event that must be in same Script If arg = "http://www.happymonkeying.com/beta/mmurltst.php" Then 'somecode Else 'somecode End if End Sub ------ |
|
|
Implemented in build 1307 and documented: http://www.mediamonkey.com/wiki/index.php/ISDBApplication::Downloader http://www.mediamonkey.com/wiki/index.php/ISDBApplicationEvents::OnDownloadFinished |
|
|
You are missing Status function. Without it there is no way to make scripts like this where OnDownloadFinish is not needed and there is no error handling in scripts. Example: ---- Option Explicit Dim UI : Set UI = SDB.UI Sub OnStartUp() Dim mnuTest Set mnuTest = SDB.UI.AddMenuItem(SDB.UI.Menu_Edit, 0, 0) mnuTest.Caption = SDB.Localize("Download Script") mnuTest.OnClickFunc = "SDBOnClick" mnuTest.UseScript = Script.ScriptPath End Sub Sub SDBOnClick(Item) SDB.Downloader.DownloadFile "http://www.spiker.wz.cz/SecretStream.mp3", "C:\aaa.mp3", False Do SDB.ProcessMessages Sleep(500) Loop While SDB.Downloader.Status("http://www.spiker.wz.cz/SecretStream.mp3") = 1 SDB.ProcessMessages End Sub ---- |
|
|
Re: Error handling: I've added the Success as second parameter, see: http://www.mediamonkey.com/wiki/index.php/ISDBApplicationEvents::OnDownloadFinished Re: Status function: Use GetDownloadProgress(), it returns -1 if the download isn't in progress, see: http://www.mediamonkey.com/wiki/index.php/ISDBDownloader::GetDownloadProgress Fixed in build 1307. |
|
|
OK what about: 1. IDLE/PAUSED and CANCELED state? 2. No handling of ERROR CODE like 404 Not Found, 403 Denied, ...? 3. No way to get Total length/size of File? |
|
|
I don't see them much useful. This is just simple API to access integrated download manager. If someone needs an additional functions like custom header sending or detailed error handling then he should use an internet access API or library like Lyricator script does. |
|
|
1. In case that there is no any status feedback scripter would need to guess downloader URL status and best solution would be remove separate PauseDownload, ResumeDownload, CancelDownload and add unified method thru Property Get/Let that will look like this adn will allow future expanding: GetURLStatus Property Get/Let Objects(URL As String) As Integer GET Results would be: -1 = URL Unknown 0 = Canceled/Failed 1 = Downloading/Active 2 = Paused/Idle 3 = Query for Resuming --- Future Use 4 = Waiting for processing (URL present in downloader queue) 5 = In process for removal from downloader queue ? = For future expansions LET Results would be: 0 = Cancel Download 1 = Download/Retry 2 = Pause 3 = Resume --- Future Use 4 = Move to end of downloader queue 5 = Remove from downloader queue (in case URL is Downloading/Active Cancel Download before removal) ? = For future expansions This way we will make things more script developer friendlier. 2. Regarding error handling I agree that we should keep it as basic as possible http://www.mediamonkey.com/wiki/index.php/ISDBApplicationEvents::OnDownloadFinished internet API is not multi platform and request advanced knowledge. Bringing internal passthru of downloader thru Success Parameter would error code (4xx Client errors along with 5xx Server errors for start) based on http://en.wikipedia.org/wiki/List_of_HTTP_status_codes, http://www.iana.org/assignments/http-status-codes and http://www.askapache.com/htaccess/apache-status-code-headers-errordocument.html along with few internal return codes -1 = Download Complete, 0 = Download Canceled. Returning these Codes would evade locking MM in Loop functions and allow Scripter to handle online content more efficiently along with ways to optimize Script code to use much less Resources (CPU) and improve overall MM speed considerably (Small scripts can be made to show resource usage). HINT: Firefox Have addon for easy live watching/capturing Headers and Responses see https://addons.mozilla.org/en-US/firefox/addon/3829/ NOTE: If it is too complicated for immediate implementation just change Success parameter to Integer instead of Boolean and give return values -1 = Download Complete, 0 = Download Canceled which will allow us later extending without compatibility issues due the fact that TRUE/FALSE represents -1/0 according to http://www.w3schools.com/vbScript/vbscript_ref_keywords.asp and proven true by this Script: ---- Dim test test = -1 If test = TRUE Then test = 0 end If If test = FALSE Then test = -1 End If ---- 3. I missed this but Length can be retrieved using http://www.mediamonkey.com/wiki/index.php/ISDBDownloader::GetURLContentLength 4. For Consistence with other Methods in Downloader http://www.mediamonkey.com/wiki/index.php/ISDBDownloader::GetDownloadProgress would be more clear if renamed to GetURLProgress ( http://www.mediamonkey.com/wiki/index.php/ISDBDownloader::GetURLProgress ) |
|
|
1. We can add the GetDownloadStatus() in a future version, but I believe that for now the GetDownloadProgress() is enough. It tells whether the download is in progress and what the progress is. 2. Ok, I extended the event and added the ResponseCode parameter, see http://www.mediamonkey.com/wiki/index.php/ISDBApplicationEvents::OnDownloadFinished 4. I think that GetDownloadProgress() is more intuitive than GetURLProgress(), but this is not a big deal. |
|
|
Verified 1333 Added Revision Tag for future easier locating |
|
|
I corrected http://www.mediamonkey.com/wiki/index.php/SDBDownloader to have reference to http://www.mediamonkey.com/wiki/index.php/ISDBApplicationEvents::OnDownloadFinished Event for easier implementation and script optimization |
