Scraping

Discuss existing features and requests for new features using [REQUEST] in the title
Forum rules
Discussions only around the RetroX application. Links to ROMs or request for them are forbidden. Insisting users will be banned
rgb
Posts: 1
Joined: Thu Apr 13, 2017 9:49 am

Scraping

Postby rgb » Thu Apr 13, 2017 10:16 am

I think that scraping could be better.
I hope that retrolux could become like plex for videogames.

This is my wish:
I'd like to organize my game and have all the possible information preferably in my language: Italian/english.
I prefer to have video preview of the game so why not import a video from emumovies or download user collection videos.
Better if all information stay outside the the roms directory; why not in my personal cloud ?
User avatar
steviebwoy
Posts: 55
Joined: Thu Apr 13, 2017 8:31 am
Location: Cornwall, UK

Re: Scraping

Postby steviebwoy » Thu Apr 13, 2017 10:29 am

Yeah, I agree. I think scraping could be a lot better. I don't mind where this information is held though, I'd be happy for it to be downloaded to my Nvidia Shield rather than the Cloud.
User avatar
fcatrin
Site Admin
Posts: 1190
Joined: Tue Apr 11, 2017 6:28 pm
Location: Quilpué, Chile
Contact:

Re: Scraping

Postby fcatrin » Thu Apr 13, 2017 2:21 pm

The main problem of scraping in general is:

a) Dependency on other's services
b) Initialization time

Currently we get the data from our own servers but we are far from covering all the games available. We do it on demand so users can start playing as soon as possible. If we get the same data from third parties we should cover the costs and we don't think that is possible now. I've reviewed some experiences about scraping from the Nostalgia and LaunchBox developers and it is not good. Don't take me wrong, it is good for the user, but a big problem for developers.

About video previews I still don't know how good could it work on Android, please check this for background info:

viewtopic.php?f=2&t=3#p16

I've been thinking about improving the way we collect the data about the games. Until now there was one person (Javier) creating all the data, covers and screenshots, but it will be a lot better if I open that part to the community so anybody can submit data for their favorite games and systems.

About cloud storage for metadata, the current implementation of public clouds (gdrive, dropbox) doesn't work as good as I thought, so we are evaluating new options on it.
Franco Catrin
RetroX developer
User avatar
steviebwoy
Posts: 55
Joined: Thu Apr 13, 2017 8:31 am
Location: Cornwall, UK

Re: Scraping

Postby steviebwoy » Thu Apr 13, 2017 9:34 pm

I'd be fully supportive of this becoming a community effort. Many hands make light work :)
XBEOX
Posts: 28
Joined: Sun Aug 27, 2017 4:12 pm

Re: Scraping

Postby XBEOX » Sun Aug 27, 2017 4:39 pm

Is this being updated? I am getting no 3DO scrapes and lots of gaps in general

Great work overall of course
User avatar
Shane_R_Monroe
Posts: 17
Joined: Mon Aug 21, 2017 6:27 am

Re: Scraping

Postby Shane_R_Monroe » Sun Aug 27, 2017 8:37 pm

I put my hand up to help as well.

Return to “Features”

Who is online

Users browsing this forum: No registered users and 14 guests