MyExperiment-ScreenScrape
This was a screenscrape of the profiles, groups, packs, files, and workflow information from the popular workflow repository site and social network for scientists myexperiment.org. All information was gathered using python scripts available on https://github.com/RichardLitt/Understanding-Workflows, and was put into…
This was a screenscrape of the profiles, groups, packs, files, and workflow information from the popular workflow repository site and social network for scientists myexperiment.org. All information was gathered using python scripts available on https://github.com/RichardLitt/Understanding-Workflows, and was put into an SQL database, and analysed with R. A paper based on this data was published:
Littauer, R., Ram, K., Ludäscher, B., Michener, W., Koskela, R. (accepted). Trends in Use of Scientific Work- flows: Insights from a Public Repository and Guidelines for Best Practices. In International Data Curation Conference, Bristol, 7-9 December, 2011.
All of this data came from the publicly available site. The urls used in scraping were gathered from the open RDF store on myExperiment.
- cc-by