Social media as passive geo-participation in transportation planning – how effective are topic modeling & sentiment analysis in comparison with citizen surveys?

Figures & data

Figure 1. Pipeline for collecting and processing data for this study

Figure 2. Volume of tweets in June 2018, categorized by both Sentiment (VADER, 1 to 5) and Tone (from IBM Watson Tone Analyzer)

Table 1. LDA results – 12 derived topics, interpretation, and percent of Tweets with that as dominant topic

TOPIC	% OF TWEETS (DOMINANT TOPIC)	INTERPRETATION	TOP CO-OCCURRING KEYWORDS
1	7%	Automated accounts for reporting service delays.	Bus, service, thanks, train, minute, late, delay, stop, strathfield, running, worst, central, newcastle, transportfornsw, min, st, update, time, sydneytrains, t1sydneytrains
2	16%	Tweet complaints related to nighttime & peak travel on trains. News events (events – sports, NYE).	Sydneytrains, hour, Australia, transportfornsw, train, service, peak, still, 2019, use, station, done, Sydney, well, macarthur, issue, night, travel, time, rain
3	3%	Users talking about issues with train services – cancellations, delays, wait times. Specific line often also specified.	Train, central, service, line, station, sydneytrains, city, minute, canceled, get, via, t1sydneytrains, platform, t8sydneytrains, Sydney, Hornsby, stop, north, trainsinfo, redfern
4	4%	News and responses – e.g. Metro, timetabling changes, new train stations.	Sydney, train, new, sydneytrains, metro, place, year, used, man, part, first, maintenance, infrastructure, area, sydneymetro, martin, lady, schofields, storm, fact
5	4%	Very short comments about train experience – air conditioning, vandalism, seating, mess. Little contextual information.	Carriage, set, number, sydneytrains, ####, heading, train, ok, always, seat, get, follow, awesome, alert, b, ####, word, inside, together, springwood
6	17%	Information and comments about cancellations, mechanical failures and strikes.	T4sydneytrains, another, bondi, trainlinksouth, future, junction, mechanical, upgrade, issue, hear, day, event, Hurstville, Cronulla, delay, world, country, much, sense, amount
7	7%	Comments on government officials and operators of transport network.	Transportfornsw, transport, public, sydneytrains, car, weekend, nswpol, gladysb, Sydney, andrewconstance, nsw, power, light, railway, system, quiet, election, rail, government, lol
8	6%	Services updates. Comments on and about platforms and carriages (including air conditioning).	Train, delay, sydneytrains, today, trip, station, trainsinfo, please, experienced, t1sydneytrains, air, platform, door, announcement, t2sydneytrains, information, t8sydneytrains, staff, time
9	13%	Short & disgruntled messages without much contextual information.	like, look, sydneytrains, town, hall, train, joke, weather, guy, good, leaving, cityrail, appreciate, get, cityfail, t1sydneytrains, thanks, sound, cheer, feel
10	2%	Users messaging responses to news about train and transport issues.	Sydneytrains, train, post, edited, auspol, rail, gladysb, get, system, locomotive, news, transit, fix, point, Sydney, like, please, service, time
11	12%	Users commenting on difficulty getting home from work. Users commenting on bus services and replacement bus services for train.	Train, bus, sydneytrains, get, work, people, one, time, back, day, driver, know, go, way, home, transportfornsw, going, need, cant, every
12	10%	Users reporting issues paying for their fare smartcard (Opal) ticketing system. News about workers’ pay/government use of funds.	Sydney, rail, sydneytrains, Opal, transportfornsw, card, nsw, train, top, state, thank, network, new, tap, railway, road, station, year, phone, police

Figure 3. Daily delayed rail and bus trips in comparison with daily volume of Sydney transport-related tweets over time; annotated with information about significant transport disruptions documented during the year

Figure 4. Daily delayed bus trips in comparison with the daily volume of Sydney transport-related Tweets

Figure 5. Daily delayed rail trips in comparison with the daily volume of Sydney transport-related Tweets

Table 2. Proportion of population in survey sample (social media) versus Labor Force and general population in Sydney

	Social Media (Twitter)		Sydney Greater Capital City Statistical Area (GCCSA)
Age Bracket	User Survey	Survey Margin (+-9.2%)	Labor Force	All Population
17 and under	N/A*	0–9.2%	2%	22%
18–24	14%	4.8–23.2%	13%	10%
25–34	41%	31.8–51%	25%	16%
35–44	25%	15.8–34.2%	23%	14%
45–54	12%	2.8–21.2%	20%	13%
55–64	8%	0–17.2%	13%	11%
65 and over	0%	0–9.2%	4%	14%

* Survey design results did not allow users under 18 to complete the survey

Figure 6. Sample of transport Tweets in Sydney by weekday and time of day

Table A1. Key Twitter search terms used in study

Tag description	Tags
Tags – Information Channels	@TrainsInfo, @BusesInfo
Tags – Transport Organization	@TransportforNSW
Tags – Individual Routes (Train)	@T1SydneyTrains, @T2SydneyTrains. @T3SydneyTrains, @T4SydneyTrains, @T5SydneyTrains, @T6SydneyTrains, @T7SydneyTrains, @T8SydneyTrains, @TrainLinkNorth, @TrainLinkWest, @TrainLinkSouth,
Tags – Individual Routes (Bus)	@BusesNorth, @BusesNorthWest, @BusesSouth, @BusesSouthWest. @BusesInnerWest, @BusesEast, @BusesWest
Hash Tags – Transport Conversations	#sydneytrains, #nswtransport, #sydneybuses, #sydneytransport, #transportfornsw

Table A2. Proportion of collected Tweets by inclusion of search term

Search Term	% Of Total Tweets
@sydneytrains	31.3%
@transportfornsw	9.5%
@trainsinfo	6.5%
@t1sydneytrains	6.0%
@t8sydneytrains	3.0%
@t2sydneytrains	2.6%
@t4sydneytrains	2.0%
@t3sydneytrains	1.8%
@trainlinknorth	1.3%
@sydneybuses	0.8%
@trainlinksouth	0.8%
@trainlinkwest	0.7%
@busesnorth	0.5%
@busesinnerwest	0.4%
@sydneytransport	0.4%
@busesinfo	0.4%
@buseseast	0.4%
@t5sydneytrains	0.3%
@t7sydneytrains	0.3%
@transportfornsw	0.3%
@nswtransport	0.3%
@busesnorthwest	0.2%
@t6sydneytrains	0.2%
@busessouth	0.1%
@buseswest	0.1%
@busessouthwest	0.1%
Total with terms	70.2%
Total results returned which didn’t include search terms	29.8%

Table B1. Survey questions

Questions

1) Please select your age range 18–24 25 – 34 35–44 45 – 54 55–64 65 +

2) Do you currently live in Sydney or within 200 km of Sydney?

3) Which of these best describes you? Full-time employed Part-time employed Looking for work Student Retired Homemaker Unable to work

4a) Have you posted about anything related to your public transport experience/its performance on the web/social media in the last 12 months? 4b) If yes above, briefly, where?

5a) Have you viewed anything related to others’ transport experience/its performance on the web/social media in the last 12 months? 5b) If yes above, briefly, where?

6) Please rate your knowledge of the following: Using a smartphone Navigating the internet Social Media Australian politics Local Government Current transport in Sydney Future plans for transport in Sydney Sydney in general Transport in general Open data

7) In the last 12 months, how have you voiced your opinion (positive, negative or suggestions) on your experience on public transport? Select as many that apply to you. Talking to my immediate social network … (family, friends and/or colleagues. online or offline) On my own personal social media profile (Facebook, Twitter, LinkedIn, Instagram) On the transport operator’s own channels/profiles on social media (Facebook, Twitter, LinkedIn, Instagram) On a member of parliament’s channels/profiles on social media (Facebook, Twitter, LinkedIn, Instagram) To an online community of interest (such as a specific page for a bus route, or public transport activism community) Through the official channels – by calling, emailing or filling out a form on transport provider’s website Complaining in person to the driver or a public transport official By attending physical community meetings or workshops The way I voted was influenced by operational transport decisions

8) When you voiced your opinion, what was the purpose? (You can select multiple) To understand – I wanted information back about a circumstance I was in To vent – I wanted to feel better about an experience by discussing with others To complain – I wanted my opinion to be used to highlight bad services To praise – I wanted people to know when and how they are doing a good job To suggest – I wanted to suggest improvements to daily public transport operations

9) When talking/posting about public transport performance in Sydney, what topics are the most important to you? Waiting Time Travel Time Crowdedness Information (or lack of information) Safety Comfort Ticketing Station/Stop Access Customer Service from staff Network design & routing Other (specify)

10) For the most important topics, how satisfied are you toward these in Sydney?

11) How satisfied are you with current services that you know of that allow you to personally contribute to the ongoing improvement public transport operations?

12) Which best describes the way you personally *want* to engage in improving everyday public transport services? Passively – I don’t want anything that I do to influence service improvements. Passively – I want data about me to be used to improve services with minimal involvement. Actively, but with no time – I want very quick ways to be able to individually contribute to service improvement. Actively, with time – I want to be involved and am willing to give my own time to improve services.

13) Can you please describe what your answer to above would look like? How does/would it work?

14) Have you used travel applications such as Uber, Ola, Didi?

(15 – Optional) Do you have any other comments, stories, anecdotes about your engagement in dialogue on public transport services? You can be as detailed as you want!

Table C1. Twitter user survey – tweeting about Transport Performance in Sydney – quantitative results

Characteristic	n = 114	%
Age
18–24	16	14%
25–34	47	41%
35–44	28	25%
45–54	14	12%
55–64	9	8%
Primary activity
Full-time employed	86	75%
Student	11	10%
Part-time employed	10	9%
Looking for work	3	3%
Other (Carer/Homemaker)	2	2%
Casual employed	2	2%
In the last 12 months, how have you voiced your opinion (positive, negative or suggestions) on your experience on public transport?
On my own personal social media profile	110	96%
Talking to my immediate social network	107	94%
On the transport operator’s own channels/profiles on social media	84	74%
Through official channels, by calling, emailing or filling out a form on transport provider’s website	43	38%
The way I voted was influenced by operational transport decisions	38	33%
On a member of parliament’s channels/profiles on social media	31	27%
To an online community of interest	27	24%
Complaining in person to the driver or a public transport official	22	19%
By attending physical community meetings or workshops	8	7%
When you voiced your opinion, what was the purpose? (You can select multiple)
To complain – I wanted my opinion to be used to highlight bad services	88	77%
To vent – I wanted to feel better about an experience by discussing with others	78	68%
To understand – I wanted information back about a circumstance I was in	62	54%
To suggest – I wanted to suggest improvements to daily public transport operations	62	54%
To praise – I wanted people to know when and how they are doing a good job	44	39%
How satisfied are you with current services that you know of that allow you to personally contribute to the ongoing improvement public transport operations?
1 – Very unsatisfied	17	15%
2 – Unsatisfied	42	37%
3 – Neither, nor	40	35%
4 – Satisfied	12	11%
5 – Very satisfied	3	3%
Which best describes the way you personally *want* to engage in improving everyday public transport services?
Actively – I am willing to take small amounts of time (a few minutes)	55	48%
Actively – I am willing to devote longer periods of time (more than an hour)	37	32%
Passively – I want data about me to be used to influence services without my input	16	14%
Passively – I don’t want anything to do with how services are designed	2	2%
Other	4	4%
How important is the following metric: Waiting Time?
1 – Not very important	1	1%
2- Not important	3	3%
3- Neither, nor	14	12%
4- Important	33	29%
5 – Very important	63	55%
How important is the following metric: Travel time?
1 – Not very important	1	1%
2- Not important	2	2%
3- Neither, nor	11	10%
4- Important	39	34%
5 – Very important	61	54%
How important is the following metric: Crowdedness?
1 – Not very important	2	2%
2- Not important	4	4%
3- Neither, nor	16	14%
4- Important	43	38%
5 – Very important	49	43%
How important is the following metric: Acce ss to information?
1 – Not very important	1	1%
2- Not important	1	1%
3- Neither, nor	9	8%
4- Important	28	25%
5 – Very important	75	66%

Figure D1. Results of LDA modeling. A darker blue color denotes a stronger relationship between co-occurring words