Web Scraping in the Statistics and Data Science Curriculum: Challenges and Opportunities

Figures & data

Fig. 1 HTML document example.

Fig. 2 HTML document with CSS example.

Fig. 3 Partial HTML tree of HTML and CSS example.

Fig. 4 Identifying the CSS selector for the table of contributions using the SelectorGadget. Retrieved on April 30, 2020.

Fig. 5 Example analysis: visualization of contributions from PACs connected to the UK to the Democratic and Republican parties over time.

Table 1 An overview of statistics and computing topics in the web scraping classroom.

	Computational topics and tools	Statistical topics and tools
Elementary	Using functions from	Data visualization
	different packages	Exploratory data analysis
	SelectorGadget
Intermediate	Writing functions	Sampling
	Iteration	Statistical inference
	String manipulation	Statistical modeling
	Text analysis
Advanced	APIs	Handling missing data
	HTML and CSS
	Hierarchical data (JSON, XML)