Web scraping for migration, mobility, and migrant integration studies
- Web scraping, a technique for extracting data from web pages, has been in use for decades, yet its utilization in the field of migration, mobility, and migrant integration studies has been limited. The field faces notorious limitations regarding data access and availability, particularly in low-income settings. Web scraping has the potential to provide new datasets for further qualitative and quantitative analysis. Web scraping requires no financial resources, is agnostic to epistemic divides in the field, reduces researcher bias, and increases transparency and replicability of data collection. As large providers of digital data such as Facebook or Twitter increasingly restrict access to their data for researchers, web scraping will become more important in the future and deserves its place in the toolbox of migration and mobility scholars. This short and nontechnical methods note introduces the fundamental concepts of web scraping, provides guidance on how to learn the technique, showcases practical applications of web scraping inWeb scraping, a technique for extracting data from web pages, has been in use for decades, yet its utilization in the field of migration, mobility, and migrant integration studies has been limited. The field faces notorious limitations regarding data access and availability, particularly in low-income settings. Web scraping has the potential to provide new datasets for further qualitative and quantitative analysis. Web scraping requires no financial resources, is agnostic to epistemic divides in the field, reduces researcher bias, and increases transparency and replicability of data collection. As large providers of digital data such as Facebook or Twitter increasingly restrict access to their data for researchers, web scraping will become more important in the future and deserves its place in the toolbox of migration and mobility scholars. This short and nontechnical methods note introduces the fundamental concepts of web scraping, provides guidance on how to learn the technique, showcases practical applications of web scraping in the study of migrant populations, and discusses potential future use cases.…
