SwissCenter - Re: IMDb SmartSearch parser released


SwissCenter

Main Menu

No account yet? Create one

Home

SwissCenter Forums

home | post reply | threaded view | help

Boardwalk :: Forum List

SwissCenter

General

<< Start < Previous 1 2 Next > End >>

Utsi
User

Posts: 135

IMDb SmartSearch parser (unofficial) - 2008/08/11 09:01 I was not completely happy with the imdb parser, so I made a modified version to fit my needs.

New features:
-Strips metadata from file names before submitting them to www.imdb.com
-If search based on file names does not return any results, use folder name instead. (Takes into account that folder structures could include "cd#"-folder and "sample"-folder).
-If folder name had to be used instead of file name in order to get good results, update the movie title so that the folder name is displayed rather than the cryptic file name (Thus enables searching for the movie based on title).
-Does not download the "Poster not submitted" image from imdb if no poster was found.

As this parser has a different name than the original imdb parser, it can co-exist in the same folder. Download and extract to "C:\Documents and Settings\All Users\Application Data\Simese\Data\ext\parsers\movie".

Scroll to the end of this thread to find the latest version.

File Attachment:
File name: www.imdb.com[SmartSearch].zip
File size:4235 bytes

Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

Hubert
User

Posts: 2

Re: IMDb SmartSearch parser released - 2008/08/11 13:05 Finally! This is the new imdb parser as far as I'm concerned. I've tried llink and a couple of others without success. Installed this one now, and it found nearly all my movies except the non-english ones. Thank you!

| | The administrator has disabled public write access.

RemBrand

User

Posts: 12

Re: IMDb SmartSearch parser released - 2008/08/26 17:21 It works like a charm, excellent work! But it doesn't seem to fetch the year, or is that just here? Transgear DVX 500E
SwissCenter on P mobile 1.6GHz / 768mb ram / Win2k + Simese

| | The administrator has disabled public write access.

Pernod

Admin

Posts: 1892

Re: IMDb SmartSearch parser released - 2008/08/26 17:25 This is the unofficial version of the parser, the official one has a fix for the year. Players : Netgear EVA700 & Popcorn A-100 (090204)
Webserver : Simese v2.5.5 - PHP v5.2.9-1 - MySQL v5.0.6.7
Server : Vista SP2 Home Premium
Spec : Intel C2Q Q6600 - 4GB RAM - 2TB HDD

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2008/08/27 07:14 I'll see what I can do about it. I'll have to diff the latest versions to find the fix. It wouldn't be a big job to integrate this functionality into the supported parser if Nigel wanted to. Then it could be turned on or off from the config page. Well, just a thought.

Edit: Here's an updated version of the parser that also gets the year.

File Attachment:
File name: www.imdb.com[SmartSearch].zip
File size:4312 bytes

Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2008/08/27 16:21 The SmartSearch just got a helluvalot smarter: It now uses the year from the filename/foldername (if existing) when submitting the moviename to imdb. That eliminates most of the duplicates and improves the search.

Please give feedback!

File Attachment:
File name: www.imdb.com[SmartSearch].zip
File size:4479 bytes

Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

RemBrand

User

Posts: 12

Re: IMDb SmartSearch parser released - 2008/08/31 06:05 Looks like it's working, thanks! Transgear DVX 500E
SwissCenter on P mobile 1.6GHz / 768mb ram / Win2k + Simese

| | The administrator has disabled public write access.

KMan
User

Posts: 143

Re: IMDb SmartSearch parser released - 2008/09/09 10:06 Utsi wrote:
The SmartSearch just got a helluvalot smarter: It now uses the year from the filename/foldername (if existing) when submitting the moviename to imdb. That eliminates most of the duplicates and improves the search.

Please give feedback!

File Attachment:
File name: www.imdb.com[SmartSearch].zip
File size:4479 bytes

Nice Parser, great work. Now if it could also support imdb.de i would be a happy man

btw, if i may make a suggestion, why dont you use regex to remove meta data,
that should be a bit quicker then your attempt i guess???
may i suggest this attempt

Code:


 function strip_title_meta ($title)
{
  $search  = array ( '/\bdvdrip\b/i'
                   , '/\bhdtv\b/i'
                   , '/\bproper\b/i'
                   , '/\bws\b/i'
                   , '/\bxvid\b/i'
                   , '/  +/g');

  $replace = array ( ''
                   , ''
                   , ''
                   , ''
                   , ''
                   , ' ');

Its just a short version, but i think you will get the idea, the \b set a word boundary making sure they only get matched if its a destinct expression and last line should remove the unnecessary spaces (i hope)

Just an idea though

Kind Regards
KMan Player : Pinnacle ShowCenter 250HD (wired)
Server : XP Pro SP2+ (Simese 1.45, SwissCenter current SVN)
Spec : Intel Pentium D830 (DualCore) 3GHz, 2GB RAM

| | The administrator has disabled public write access.

Pernod

Admin

Posts: 1892

Re: IMDb SmartSearch parser released - 2008/09/09 11:21 Now if it could also support imdb.de
Why? From what I've seen at imdb.de the only difference is that the genres are in German. All synopsis are still in English. Players : Netgear EVA700 & Popcorn A-100 (090204)
Webserver : Simese v2.5.5 - PHP v5.2.9-1 - MySQL v5.0.6.7
Server : Vista SP2 Home Premium
Spec : Intel C2Q Q6600 - 4GB RAM - 2TB HDD

| | The administrator has disabled public write access.

KMan
User

Posts: 143

Re: IMDb SmartSearch parser released - 2008/09/09 12:18 Pernod wrote:
Now if it could also support imdb.de
Why? From what I've seen at imdb.de the only difference is that the genres are in German. All synopsis are still in English.

because imdb.com doesn't reply on all german titles
e.g.
Die spezialisten unterwegs (Misfits of Science) on imdb.com and Die spezialisten unterwegs (Misfits of Science) on imdb.de

First one gives no hit, second does
but i'd admit adding ;site=aka to the imdb.com query would also bring up that hit, though with several alternate titles this might end up in a mess, so for me matching against only german alternate titles seems the better option ?

maybe it would even be possible to handle it language dependent like in the thetvdb.com parser

hope this clears things up

Kind Regards

KMan Player : Pinnacle ShowCenter 250HD (wired)
Server : XP Pro SP2+ (Simese 1.45, SwissCenter current SVN)
Spec : Intel Pentium D830 (DualCore) 3GHz, 2GB RAM

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2008/09/09 14:39 KMan: Thanks for your input. German is out of my scope as I've only added functionality that I need myself

If you've got time, you could play around with it and see if you can figure it out.

About your code suggestion, there would be a problem unless you included every possible string in your search array. The reason is that most names are like something like this: "Dude Wheres My Car 2000 720p BluRay x264 Utsi". So if you didn't have "Utsi" in your array, the search would look like this: "Dude Wheres My Car Utsi (2000)". My code chops off everything after the first finding (which in this case is " 2000").

I don't know much about regular expressions and php, but if you're familiar with it, you're free to modify the code and post a new version. Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

KMan
User

Posts: 143

Re: IMDb SmartSearch parser released - 2008/09/09 15:18 Utsi wrote:
KMan: Thanks for your input. German is out of my scope as I've only added functionality that I need myself

If you've got time, you could play around with it and see if you can figure it out.

About your code suggestion, there would be a problem unless you included every possible string in your search array. The reason is that most names are like something like this: "Dude Wheres My Car 2000 720p BluRay x264 Utsi". So if you didn't have "Utsi" in your array, the search would look like this: "Dude Wheres My Car Utsi (2000)". My code chops off everything after the first finding (which in this case is " 2000").

I don't know much about regular expressions and php, but if you're familiar with it, you're free to modify the code and post a new version.

ups, sorry, so as you noticed i dont have any knowledge of php at all!
didnt realize you didnt only replace your matches.

from what i can tell to find german titles aswell this

Code:


     $site_url    = 'http://www.imdb.com/';

would need to be changed to

Code:


     $site_url    = 'http://akas.imdb.com/';

but i have now idea how the

Code:


     
1.    
Misfits of Science (1985) (TV)
 aka "Spezialisten unterwegs, Die" - Germany

aka part need to be matched.

so i know what to do, but dont have a clue how to do so....

Maybe you find some spare time to check it out if you are in the mood, i'd highly appreciate it...

But i can understand if you dont want to spend any time on it, since it is of no use to yourself, aswell!

Kind regards

KMan Player : Pinnacle ShowCenter 250HD (wired)
Server : XP Pro SP2+ (Simese 1.45, SwissCenter current SVN)
Spec : Intel Pentium D830 (DualCore) 3GHz, 2GB RAM

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/05 07:29 Heres an improved version of the SmartSearch parser. Enjoy!

Background:
I noticed that IMDB is extremely picky when it comes to case sensitivity. Therefore, I had to modify the parser so that it doesn't use upper case/lower case where it's not supposed to. That makes "Star Wars Episode Iii Revenge Of The Sith" -> "Star Wars Episode III Revenge of the Sith" which IMDb accepts.

Further more, IMDb doesn't always like to get the year as a parameter. So I modified the code to do the following:
1) Use title + year.
2) Use folder name + year.
3) Use title.
4) use folder name.

I'm happy to say that the parser can now tell the difference between "Assault on Precinct 13 (2005)" and "Assault on Precinct 13 (1976)" given the files "Assault.On.Precinct.13.1976.720p.BluRay.utsi.mkv" and "Assault.On.Precinct.13.2005.720p.BluRay.utsi.mkv".

Also, I added a couple of tags to the metadata array and did some clean-up in the code. Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/05 07:36 Here's the latest version. Right-click and select save as <Whatever>.zip

File Attachment:
File name: www.imdb[SmartSearch].php.zip.
File size:4975 bytes

Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/05 10:23 Found a typo that in some rare cases could mess up the search. Corrected.

File Attachment:
File name: www.imdb[SmartSearch].zip
File size:4992 bytes

[/file] Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

sdeitsch
User

Posts: 55

Re: IMDb SmartSearch parser released - 2009/01/13 16:30 Firstly I'd like to thank you for your work. This is the BEST parser I've used on SC and I've tried them all and even tried to customize them.
This is perfect for me, as I have always named my files according to IMDB naming convention Title (DATE) and the other parsers always screwed up on versions of movies with the same title because they did not take the date into account.... you are a genius!

That being said, I have a favor to ask. IMDB has no rhyme or reason to the ordering of their cast lists. Sometimes it's by appearance, or billing or whatever and sometimes the actual stars of the film end up on the 2nd page of actors requiring the user to click 'more'
as an example, when using your parser for Benjamin Buttons, Brad Pitt wasn't even on the cast list!!!!

Is there a way to always capture and store the ENTIRE cast list as presented when the user clicks 'more' that would save me a buttload of manual entry work.

Another example is that Alan Rickman (snape) is not in any harry potter films because his name is always on the next page.

I really appreciate what you've done to date and this would be the icing on the cake.

Thanks again

sdeitsch
PIII 3.0Ghz 1Gb RAM
WinXP SP3
IOData Avel Linkplayer2 (2)
Latest build SC

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/14 14:15 Hey sdeitsch,
thanks for your feedback. I've done my best to make the SmartSearch a good parser for everyone, whether you're using the regular naming convention or the scene naming convention as I am. I did some experimenting to get the year bit right, and I'm happy with the results. I wouldn't go as far as calling myself a genius, though, but if you insist

Your request is a good one, I agree that it's an absolute outrage! But I will have to put it on my todo list as I don't have as much spare time today as I had a few weeks ago. I believe the solution is already in the application, it's already clicking on links so it should be as simple as a copy/paste job with some small modifications. Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

avgjoemomma
User

Posts: 71

Re: IMDb SmartSearch parser released - 2009/01/15 19:31 Utsi, this is really awesome. I found a few small bugs.

If the Movie file name has an ' then the letter after that is capitalized by accident. Example:

Filename: Logan's Run.avi

In Swisscenter the filename shows as Logan'S Run

Sometimes if the title is not found you default to the name of the folder which in my case is Movies

Player: Popcorn Hour A-110
Router: Linksys-Cisco WRT54GL (DD-WRT) (TCP Vegas enabled)
SC server: QNAP TS-109 II Apache v1.3.37 MySQL v5.0.27-log
NAS Server: QNAP TS-109 II

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/16 02:06 Hi avgjoemomma,
Thanks for reporting this, I wouldn't have noticed otherwise.

When I get the time I'll do some debugging and maybe some regexping to fix the captialization problem.

I'll have to make an exception for movies located in folder named "Movies" as searching www.imdb.com for "Movies" returns hits for an actual movie called "Movies (1999)". As a temporary workaround you could name your movies folder something else that doesn't return any results from imdb

Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

avgjoemomma
User

Posts: 71

Re: IMDb SmartSearch parser released - 2009/01/16 13:27 Thanks again, Utsi

Player: Popcorn Hour A-110
Router: Linksys-Cisco WRT54GL (DD-WRT) (TCP Vegas enabled)
SC server: QNAP TS-109 II Apache v1.3.37 MySQL v5.0.27-log
NAS Server: QNAP TS-109 II

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/19 01:55 Here's a quick progress report:

-I've fixed the captitalization of "'s" problem.
-made the parser get all the actors (page1 + page2).
-made a function to make sure that the parser doesn't look for movie based on folder names for folder names typically containing collections of movies. The list so far: "Movies", "Movie", "Films", "Filmen", "Filmer". Any other suggestions on folder names to include in the list?

I'll do some testing after work today and submit it when I'm done. Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/19 13:49 OK,
Thank you guys your feedback helping me make the SmartSearch parser the most comprehensive tool for getting the right info for our movies without any manual work!

SmartSearch does now get all the actors for every movie.

If you prefer having all your movies in one folder or collections of movies in the same folder, you can simply name your folders "Movies", "Movie", "Films", "Filmen", "Filmer", "Videos" or "Video" and the parser won't try to get movie details based on the folder names if nothing is found based on the file name itself.

Please let me know if it works as expected! Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/19 13:54 And the file.. (Right-click and save as "www.imdb.com[SmartSearch].zip)

File Attachment:
File name: www.imdb[SmartSearch].zip.
File size:5324 bytes

Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

admin

Admin

Posts: 1880

Re: IMDb SmartSearch parser released - 2009/01/19 14:27 How about adding str('VIDEO') and str('MOVIE_OPTIONS') to the list so that the values for "Video" and "Videos" are picked up for the user's language? Server : Windows Home Server (HP MediaSmart EX470)
Player #1: Pinnacle Showcenter 200 (wired ethernet)
Player #2: Popcorn Hour A110 (wired ethernet)

| | The administrator has disabled public write access.

avgjoemomma
User

Posts: 71

Re: IMDb SmartSearch parser released - 2009/01/19 14:38 Utsi, this is amazing. Perfect! This should be standard

Player: Popcorn Hour A-110
Router: Linksys-Cisco WRT54GL (DD-WRT) (TCP Vegas enabled)
SC server: QNAP TS-109 II Apache v1.3.37 MySQL v5.0.27-log
NAS Server: QNAP TS-109 II

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/19 15:12 Sweet!

@Admin: Great suggestion, I'll add them to the list.

Here's the latest version.

File Attachment:
File name: c1e8771ea3afc769555f7adaa2fcfa0d.
File size:5343 bytes

Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

Stu2j

User

Posts: 2020

Re: IMDb SmartSearch parser released - 2009/01/19 16:35 SmartSearch does now get all the actors for every movie
Interestingly enough, this is one of the things I have always preferred about dvdloc8. The list of actors picked up by imdb can sometimes take an entire page and all I really want is the major players.

Look up Iron Man on dvdloc8 will give you this:

Jeff Bridges
Terrence Howard
Samuel L. Jackson
Robert Downey Jr.
Gwyneth Paltrow
Hilary Swank

Look up Iron Man on imdb and the list of actors will be well over 50 including such well knowns as who played "lady at craps table" and "kid in suv"

Just an observation that sometimes more isn't better.

FWIW.. Showcenters 1000g/200/250HD Popcorn Hour A-100
Netgear HDX101 Powerline Adapters
Windows XP SP3 Simese 2.5.5 Beta SwissCenter latest SVN
NAS Server Intel PD 2.66Ghz/2GB Mem/1TB Storage
Hauppauge HVR 1600 PVR

| | The administrator has disabled public write access.


Utsi Visitor	Re: IMDb SmartSearch parser released - 2009/01/20 02:08 I doubt that "lady at craps table" would agree with you I agree that 50 actors for one film is too much, but still, it's better than just a random selection of some of the actors. On the other hand I bet the biggest movie enthusiasts would want to know who played the kid in suv. What could solve this would be to create a customizable cross site parser. But that would have to be done by Nigel or Rob as we would need access from the config page. You could have checkboxes and radiobuttons where you could decide where to get the actors, where to get the images, directors, whether to use folder names and whether to use scene naming standards and so on. Much of the functionality I've added here could be moved to the core of SC and used for all parsers. I don't think this is anything they'll prioritize and I don't blame them, they're adding so much other cool stuff.
	\| \| The administrator has disabled public write access.

avgjoemomma
User

Posts: 71

Re: IMDb SmartSearch parser released - 2009/01/23 15:26 Utsi, here's another idea to see if you can get it implemented. One of the problems with the synopsis from IMDB is that the plot as showed on IMDB is cut off. If we can get the synopsis from http://www.imdb.com/title/ttxxxxxxxx/plotsummary then it would be complete. Hope that makes sense. Player: Popcorn Hour A-110
Router: Linksys-Cisco WRT54GL (DD-WRT) (TCP Vegas enabled)
SC server: QNAP TS-109 II Apache v1.3.37 MySQL v5.0.27-log
NAS Server: QNAP TS-109 II

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/24 04:47 I've already considered it, and found that the plot summary often is too much for one page. Look at this example: http://us.imdb.com/title/tt0076759/plotsummary
That's why I haven't done it. Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

sdeitsch
User

Posts: 55

Re: IMDb SmartSearch parser released - 2009/01/25 01:11 Ok....
Firstly, THANK YOU UTSI FOR EXTENDING THE ACTORS LIST!!!! WOOHOOO!!!!
You have saved me a LOT of cut and paste.

So I got overexcited and cleaned out my database to completely rebuild it with the new parser aaaaannnnnddd.....
I have a problem

As I mentioned earler I name all of my files according to IMDB convention ( in hopes that this would reconcile any issues in parsing ) and yet the current parser is not delineating between multiple versions of a file.
As an example:

Dracula (1931).avi, Dracula (1958).avi and Dracula (1992).avi are all named simply Dracula and the year the parser gives for ALL of them is 1992.

Same thing with my other remakes.

Is there some way to include my parenthetical year in the search parameter?
That would do the trick!
thanks Utsi for all of your work so far and thanks in advance for whatever you can do.

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/25 05:54 Sorry to hear about that. I've created a folder with the files you mentioned to test it myself, and I've been getting the same results. It appears that SwissCenter is removing info embedded in "(" and ")" before sending it to the parser. It works if you're using the form "title.year.avi" for now. Will have to see if I can manually dig into the file name to get the year in this case.

I'll report back when I've had the chance to do further investigation! Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

sdeitsch
User

Posts: 55

Re: IMDb SmartSearch parser released - 2009/01/25 11:34 Thank you so much!!
This is GREATLY APPRECIATED!!!
I'll keep tabs on your updates....
thank you thank you

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/25 15:38 OK,
I think I've fixed it, or at least it works for me. I'm using Linux, I'm not sure if that will make any difference if you try it with Windows.

Please test and report your results!

File Attachment:
File name: www.imdb.com[SmartSearch]_00001.zip
File size:5544 bytes

Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

hispanico

User

Posts: 207

Re: IMDb SmartSearch parser released - 2009/01/26 04:45 Utsi wrote:
OK,
I think I've fixed it, or at least it works for me. I'm using Linux, I'm not sure if that will make any difference if you try it with Windows.

Please test and report your results!

File Attachment:
File name: www.imdb.com[SmartSearch]_00001.zip
File size:5544 bytes

I installed last version 1.22...i can use this or is already included in update ?

Thank Player: Showcenter 250HD
Server1: Windows Vista SP1 PC
3tb 2big quadra lacie
Webserver : Simese v2.5.5
PHP : v4.4.4 - MySQL : v5.0.67
Asus P5K3 Deluxe - Intel E8600 - DDR3 4GB

| | The administrator has disabled public write access.

avgjoemomma
User

Posts: 71

Re: IMDb SmartSearch parser released - 2009/01/26 14:19 Utsi wrote:
I've already considered it, and found that the plot summary often is too much for one page. Look at this example: http://us.imdb.com/title/tt0076759/plotsummary
That's why I haven't done it.

Hmm, yeah that is really too long. Amazon has nice summaries but that's a lot of work to implement

Player: Popcorn Hour A-110
Router: Linksys-Cisco WRT54GL (DD-WRT) (TCP Vegas enabled)
SC server: QNAP TS-109 II Apache v1.3.37 MySQL v5.0.27-log
NAS Server: QNAP TS-109 II

| | The administrator has disabled public write access.

sdeitsch
User

Posts: 55

Re: IMDb SmartSearch parser released - 2009/01/28 12:25 sorry Utsi....

didn't work.... still one 6+ hrs long dracula movie and now, as if that wasn't bad enough, I think I have TOO MANY ACTORS now because I CANNOT browse by actor, it times out everytime.....D'OH

I'm gonna keep trying things and keep you apprised, thanks for all you do.

Win XP sp3
P4 3Ghz
Linkplayer2

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/28 13:19 Sdeitsch,
I can't see why it fails for you! I've created 3 dummy dracula files by copying the file names you provided into my file system and run it through the parser and it worked great, even under Windows. Check out the attached zip file.

Now, did you do a proper update by removing your media locations and running a full scan before adding them back and re-scanning? Have you made sure that the parser is in the correct folder. You did delete the old dracula.jpg's? Have you tried manually doing a scan for one of the older dracula movies?

If none of the above works, please post a copy of your file structure like I've done here so I can test it myself.

How many actors have you got?

File Attachment:
File name: Movies.zip
File size:106724 bytes

Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

sdeitsch
User

Posts: 55

Re: IMDb SmartSearch parser released - 2009/01/28 13:37 First, I just want to thank you again for your help and concern.

Now, I'll tell you what I did, and you tell me how stupid I am

First I used the Load/Save settings option to backup my config.
Then I used the Create Database option to create a new empty db
Then I restored my settings with the Load/Save Settings option thereby retrieving my Media Locations and Categories lists.
I then searched for new media by Category (in this case 'Films')
I did NOT delete any jpegs

I have a question. When you say it worked, what are the titles of the movies post update, because ultimately the search seems title-based and the title for all three of these is returned as simply Dracula with no additional info.

and ..... How do I find out how many actors there are.. sql?
like...
select count from actors
or something like that?
I'm ignorant, but quick so I'm yours to instruct

thanks again

Win XP sp3
P4 3Ghz
Linkplayer2

p.s. with the actor thing, do you know of a way to simply extend the timeout period to give the system more time to produce the actors list?
tia

| | The administrator has disabled public write access.

admin

Admin

Posts: 1880

Re: IMDb SmartSearch parser released - 2009/01/28 14:03 and ..... How do I find out how many actors there are.. sql?
like...
select count from actors
Almost...
select count(*) from actors

p.s. with the actor thing, do you know of a way to simply extend the timeout period to give the system more time to produce the actors list?
There is a setting in your php.ini - the default is 30s Server : Windows Home Server (HP MediaSmart EX470)
Player #1: Pinnacle Showcenter 200 (wired ethernet)
Player #2: Popcorn Hour A110 (wired ethernet)

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/28 14:07 I don't think you're stupid at all

I've never used the save/load settings function. I suggest you simply go to config-> media locations and remove your movie folders. Then do a full search for films so that it forgets all about your movies. Then go back to media locations and add your moive folders. You can delete all your dracula covers from windows explorer before doing a new movie search for films. That will give your the right covers.

My titles were all "Dracula", but when I select it when browsing by title, I get the info "3 titles" and the choice to refine by year and so on, so I can navigate to the right one. I'm using Swiss Center SVN Revision [1164] by the way, so maybe you need to update your version as well.

To find the number of actors, go to "edit database" and type: select count(*) from actors

I have 4147

Hope this helps! Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

sdeitsch
User

Posts: 55

Re: IMDb SmartSearch parser released - 2009/01/28 15:00 actor count = 100838 whooaaaaa

I have made some tweaks to a few files in the past, I am running the latest release, but perhaps one of my customizations is responsible for this...
When I select Dracula I get the info and picture for the 1931 version and a running time of 6h 23m 55s

my choices on that page are: play now, add to playlist, lookup in wikipedia and video info. I am not given refine options.
Maybe I should just reinstall?
Another question, the jpgs I have for these movies are all correct as I got them myself previous, does having these pics already in any way negatively affect the parsing?

thanks again for everything and for the admin sql and php lessons

Win xp sp3
P4 3Ghz
Linkplayer2

| | The administrator has disabled public write access.

admin

Admin

Posts: 1880

Re: IMDb SmartSearch parser released - 2009/01/28 15:39 actor count = 100838 whooaaaaa

Eeek!!! That's distinct actor's - ie: all unique.

There's another table actors_in_movie that contains the mapping of each actor to one or more films. I'm guessing that table is 10x bigger (at least).

There's no wonder that it's all going a bit slow.
Have you tried enabling the "cleanup database" option when you do a media search? It will remove any actors that are no longer referenced in any of your films. Server : Windows Home Server (HP MediaSmart EX470)
Player #1: Pinnacle Showcenter 200 (wired ethernet)
Player #2: Popcorn Hour A110 (wired ethernet)

| | The administrator has disabled public write access.

sdeitsch
User

Posts: 55

Re: IMDb SmartSearch parser released - 2009/01/28 16:17 yeah... that's a lot of actors

and I did the db cleanup thing last night hoping it would solve the other issues....
I'm going to do a complete clean install and let everyone know how it goes and how many actors I have.

incidentally I have 4186 films

| | The administrator has disabled public write access.

Stu2j

User

Posts: 2020

Re: IMDb SmartSearch parser released - 2009/01/28 16:29 admin wrote:
actor count = 100838 whooaaaaa
Eeek!!! That's distinct actor's - ie: all unique.
I knew this was going to be a problem (see above) with picking up every actor for every movie. The IMDB list can be over 100 unique actors for some movies. When you start listing out who played prisoner #1, prisoner #2, etc the database is bound to grow exponentially. Even worse, some actors are listed many times such as Samuel L. Jackson and Samuel Jackson etc. Showcenters 1000g/200/250HD Popcorn Hour A-100
Netgear HDX101 Powerline Adapters
Windows XP SP3 Simese 2.5.5 Beta SwissCenter latest SVN
NAS Server Intel PD 2.66Ghz/2GB Mem/1TB Storage
Hauppauge HVR 1600 PVR

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/28 17:02 Darn!

If this problem persists, I'll revert the actors change as soon as possible. Does anyone else have the same problem? I only have ~100 movies, so for me it doesn't represent any problem. Yet.

Sdeitsch, I recommend you upgrade to the latest development build. I don't know if several movies with the same title is supported in older versions. That can be done in the config screen from your player.

I was only suggesting that you removed your covers as it would more easily prove that the parser works.

I also recommend you only add a small portion of your movies at first to test as it'll probably take all night to parse over 4000 movies! Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

sdeitsch
User

Posts: 55

Re: IMDb SmartSearch parser released - 2009/01/28 18:13 I really want that 2nd page of actors and I am hoping that this issue is unique to me.
This evening I will be doing a clean install, upgrade to svn and add a single drive of films to 'see' I will gladly report my findings
----------------------------
edit
ok I rebuilt everything and the parser works and doesn't
firstly:
movie count 4125
actor count 98301

The descriptions are as you said, for dupe titles I am presented the option to refine by genre, year etc. The info is correct for the version; however, the jpgs are frequently only for the latest version.
Also there is something about this parser that has 13 of my movies with no title at all.
And my browse by actor still times out.
Thanks for all of your hard work,

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/01/31 17:13 The jpg issue is an easy one: The parser does not delete covers that are already there, you need to manually delete the old covers before doing a search, otherwise the parser assumes the existing jpg's are the right ones.

Can you give me more details about the movies that shows up without a title, could you provide file names and full paths so I could test it for myself?

The timeout is a beyond me, maybe if you exported your database and uploaded it to one of the Gold Developers they could do some performance testing and see if some optimization could be done. You did increase the timeout variable in the config file? Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

sdeitsch
User

Posts: 55

Re: IMDb SmartSearch parser released - 2009/02/01 23:24 I just assumed the timeout was because I had way too many actors for the db to handle, that's why I was asking if I should use a better db or webserver. I changed my timeout settings in php.ini but to no avail.

Anyway my significant other was getting peeved about not being able to watch the system that I finally just blanked out the db, pointed to the supported imdb parser and reset it.
So now I have 4260 movies and 30654 actors and the system is humming beautifully.
And I don't have my second page of actors and my dupes are linked and all of this will require my manual edits as they did in the past, just as Brad Pitt will not be in Benjamin Button and Alan Rickman will not be in any Harry Potter film.

Is there a better site to parse for a proper cast list? With ONLY the stars of the picture?
----------------------------------------------
edit
I still LOVE this parser!!! At least I can manually use the smartparser for problem titles and that saves a LOT of work and I thank you thank you

| | The administrator has disabled public write access.

Utsi
User

Posts: 135

Re: IMDb SmartSearch parser released - 2009/02/02 01:39 Sounds like something else timed out, could be the db connection perhaps. It should be possible to configure the system to wait for hours if neccessary, even though your significant other will most likely time out before that.

If you're willing to spend some time in front of your computer, you could try installing Linux with Lighttpd. I'm running Ubuntu which comes with dual boot by default so you can still keep Windows. The performance of my SwissCenter sky rocketed when I did this.

Imdb is the best site we've got today. dvdloc8 has a better actor list, but they're pissy about getting too many requests from the same ip so they'll just block you. Player: Popcornhour A-100
Server: Ubuntu 9.04, lighttpd/fastcgi
System specs: Intel Pentium 4 CPU 2GHz, 756MB RAM

| | The administrator has disabled public write access.

<< Start < Previous 1 2 Next > End >>

Boardwalk :: Forum List SwissCenter General

Simpleboard Forum Component 1.1.0 Stable
Two Shoes Mambo Factory

Screenshots

www.flickr.com

This is a Flickr badge showing public photos from swisscenter. Make your own badge here.

Mambo is Free Software released under the GNU/GPL License.