Encoding problem with movie names - 2008/11/02 06:42hello all,
i'm facing a big problem. i'm running swisscenter 1.21.1 on a linux (gentoo) host and nearly all things are working fine. but one thing is making a lot of trouble to me. its an encoding problem i'm facing with data that is written to the database.
for example i've got the movie Arsène Lupin. when i manualy put in this movie name, swisscenter is cutting the name before the special character thus only Ars gets written to the database. if i change the movie name manualy at the database, at the configuration page - edit movie details - the name is shown as Arsène Lupin.
this way it is impossible to manage this film. its also impossible to use the movie database lookup to gain the movie's details from the internet.
i figured out that all the pages generated by swisscenter has got the Windows-1252 character set defined. is there a specific reason to do this? i've never seen a web page that is using this encoding. is it possible that this is causing my problems?
the problem is quite funny. as i got the german language pack activated, i did not figured out any character encoding problem within texts at the config pages of swisscenter. the only thing that is not working are things like the movie title as mentioned above. i think the problem occures with all fields that are editable and that are being written/looked up to/from the database. i believe this because adding a regie entry with a special character in ended up in having no regie entry set at all after the click to the save button.
there are other kinds of special characters that do not work too. e.g. Æon Flux is also not working. note the special AE!
have you got any idea what the problem is and how to solve it?
my swisscenter database is set up using the utf8_general_ci kollation and as i'm able to store the special characters using phpMyAdmin, i do not think that this is a database problem. i guess its a problem in accessing the database.
thank you very much.
greez,
dialsc
| | The administrator has disabled public write access.
dialsc
User
Posts: 15
Re: Encoding problem with movie names - 2008/11/02 09:27hello,
just an update. german umlaute also do not work!
| | The administrator has disabled public write access.
Pernod
Admin
Posts: 1504
Re: Encoding problem with movie names - 2008/11/02 10:10We have many users that require special characters in their local language that are not having problems. So i'd say that you have something not setup correctly.
The Windows-1252 encoding is used for all pages sent to the media players. This is a common encoding that we know all players will accept. I believe some players don't like utf-8 so until we have unique configuration for each player type then we will use Windows-1252.
How did you setup your database? It's supposed to be latin1, and SwissCenter writes to the database converting characters accordingly. If your database is utf8 then characters will be converted incorrectly. We do plan to update the database to utf8 in the future though.Player : Netgear EVA700 & Popcorn A-100 (081224) Server : Vista SP1 Home Premium (Simese 2.06, SwissCenter current SVN) Spec : Intel C2Q Q6600, 2GB RAM, 2Tb storage.
| | The administrator has disabled public write access.
dialsc
User
Posts: 15
Re: Encoding problem with movie names - 2008/11/02 10:23hello,
thanks for the answer. it now makes sence to me that the page encoding is set to Windows-1252. at least i understand it now, thank you.
i created my database and only the database with phpMyAdmin and defined the kollation mentioned.
expacting latin1 at the database could be an explanation for the problem. do think it would be a good idea to convert the database to latin1? if you do, what latin1 would be the one to use? latin1_bin, latin1_*_ci?
greez,
dialsc
| | The administrator has disabled public write access.
Pernod
Admin
Posts: 1504
Re: Encoding problem with movie names - 2008/11/02 10:49I believe we use the default collation method latin1_swedish_ci.
When SwissCenter was originally developed (a few years ago) our knowledge of MySQL was limited so default options were used.
When (if) we restructure the database we'll probably convert to utf8 and use the innodb engine, currently myiasm.Player : Netgear EVA700 & Popcorn A-100 (081224) Server : Vista SP1 Home Premium (Simese 2.06, SwissCenter current SVN) Spec : Intel C2Q Q6600, 2GB RAM, 2Tb storage.
| | The administrator has disabled public write access.
dialsc
User
Posts: 15
Re: Encoding problem with movie names - 2008/11/02 11:11i totaly deleted swisscenter from my harddisk and also the whole database at the mysql server.
i than created a new database from phpMyAdmin having the latin1_swedish_ci kollation set. i reinstalled swisscenter and rerun the databaseCreation task.
no success! the problem is the same!
any other idea?
thx, greez,
dialsc
| | The administrator has disabled public write access.
dialsc
User
Posts: 15
Re: Encoding problem with movie names - 2008/11/02 15:54hello,
i got a first glue on how to solve the problem. i edited ./config/config_movie.php and changed a part of the form definition for the update_single action.
the special characters are getting written correct to the database by using this form definition. unfortunately they're not getting show corrent at the swissCenterConfigPage(s) afterwards. the value of the title form field is also filed with a wrong value afterwards: Arsène Lupin
maybee this will give you a hint?
greez,
dialsc
| | The administrator has disabled public write access.
dialsc
User
Posts: 15
Re: Encoding problem with movie names - 2008/11/02 15:58hi,
more infos from here...
i did set up the following configuration parameter: CONFIG_PAGE_CHARSET=UTF-8
now the movie's title is drawn correct, YES. the value in the form field title is also drawn corret.
the only thing that is not working are all the special characters in the page texts from the language file. For example "Zurück zur Liste" gets "Zur�ck zur Liste".
| | The administrator has disabled public write access.
dialsc
User
Posts: 15
Re: Encoding problem with movie names - 2008/11/02 16:02i removed the modified form definition mentioned above and it still works. both drawing and saving seams to work.
maybee it is of interesst, that my host is configured to use utf-8 all over. even the mysql daemon settings are saying to use utf-8 for a lot of configuration parameters.
| | The administrator has disabled public write access.
dialsc
User
Posts: 15
Re: Encoding problem with movie names - 2008/11/02 16:06setting PLAYER_PAGE_CHARSET=UTF-8 fixed the problem, that the special characters are not getting drawn correct at the frontend movie page(s). but the problem with the language file texts occurs now.
| | The administrator has disabled public write access.
dialsc
User
Posts: 15
Re: Encoding problem with movie names - 2008/11/07 08:46hello all,
what will happen with this problem. i assume that this problem affects more than just me because it has to. in cannot imagine that things are working fine for others regarding e.g. french special characters.
can anyone confirm such a problem?
in my opinion having the settings changed to utf-8 and seeing that (allmost) all things are working is the proof that this encoding problem cannot work with the encoding used currently (windows-****).
it's realy a bummer that i'm facing this problem as i like swisscenter.
for sure there are things that do not work proper now, after having chagned the encoding settings. e.g. the internetMovieDb is not working with the name having the special french character within but this is just another encoding problem that could be fixed the same way.
can we expect you to switch to a decent encoding like utf-8 for the whole application?
thank you very much,
greez,
dialsc
| | The administrator has disabled public write access.
Pernod
Admin
Posts: 1504
Re: Encoding problem with movie names - 2008/11/07 09:00can we expect you to switch to a decent encoding like utf-8 for the whole application? Yes, it's on the list of things to do here.
We have many German, Dutch, Italian users that have not mentioned this problem though, so it's not high priority at the moment.Player : Netgear EVA700 & Popcorn A-100 (081224) Server : Vista SP1 Home Premium (Simese 2.06, SwissCenter current SVN) Spec : Intel C2Q Q6600, 2GB RAM, 2Tb storage.
| | The administrator has disabled public write access.
dialsc
User
Posts: 15
Re: Encoding problem with movie names - 2008/11/07 09:25hi,
i guess that these people do the same thing i normaly do. they avoid using special characters at any place. no umlaute in file names at the hard disc, nor french special characters in movie titles and so forth.
but i encountered the problem when i was looking for arsene lupin. the problem was, that without the è (Arsène Lupin) the internetMovieDatabaseLookup does not work -> does not find the movie. therefore i was trying the "correct" movie title which got me facing the encoding problems.
in my opinion it should be higher priorized to get swisscenter use utf-8 at all! but i know, everybody wants his problem to be fixed with a high prio...
anyway, this should realy be done soon. i, for example, do not work with swisscenter anymore until this has been fixed, sorry...
greez,
dialsc
| | The administrator has disabled public write access.
dialsc
User
Posts: 15
Re: Encoding problem with movie names - 2008/11/07 09:34by the way, i do not think that changing the database to utf-8 only will do it at all. i realy think that the whole application should be using utf-8. this encludes the page-encoding of all html pages as well as all the other places where encoding is of any matter.
| | The administrator has disabled public write access.
Stu2j
User
Posts: 1380
Re: Encoding problem with movie names - 2008/11/07 22:42the problem was, that without the è (Arsène Lupin) the internetMovieDatabaseLookup does not work -> does not find the movie.
Just as a quick note.. IMDB can find any movie by adding the IMDB code the file name, for example:
moviename [tt0373690].avi
Anything in brackets will be stripped out when the movie title is added to the database.Showcenters 1000g/200/250HD Popcorn Hour A-100 Netgear HDX101 Powerline Adapters Windows XP Simese 2.07 running latest SVN NAS Server IBM NetVista 900Mhz/256RAM
| | The administrator has disabled public write access.
Forums 
