White paper originally published June 2015.

Sample datasets can be found here.

 

The question: “In an age of social media, do comments sections still have value?” This is the question posed by Chantal Braganza in J-Source.ca, the Canadian Journalism Project, regarding the ongoing problems news sites have had with vitriolic comments and flame wars in media sites’ comment sections.

As a regular bicycle commuter, I see in real life the interactions — often negative — between cyclists and drivers that must share the roads. When I read an article in any news source about anything to do with cycling or infrastructure, I tend to avoid the comments because they always, inevitably, devolve into insults and negative voices. So I wanted to find out what those conversations look like on Twitter.

• Do people express as much anger?
• Are the reactions more positive?
• Does Twitter have any effect on legislation and policy?

Here are the questions I sought to answer:

• What kinds of conversations are happening in the cycling community via Twitter, through replies and retweets?
• Are these interactions happening between the cycling organizations and their supporters, or is there social engagement happening outside of those circles?
• How does this differ from news articles?
• And, ultimately, when it comes to cycling, what do people talk about on Twitter?

To set a baseline, I decided to use the Cascade Bicycling Club (@CascadeBicycle) as the center point for creating my datasets. Cascade has a large sphere of influence in the Seattle area’s cycling community. The non-profit organization creates programs that engage and encourage cyclists of all ages, and it lobbies locally and statewide in favor of infrastructure and policy to benefit these cyclists alongside other types of transportation.

• ~10,800 Twitter users follow Cascade on Twitter.
• Cascade has 3,560 Friends, which means it follows that number of users.

At the same time, I wanted to look outside of the cycling community to see what types of conversations were happening about cycling there, just to get a sense of how people who don’t fall within the Cascade sphere talk about cycling. Clearly the conversations don’t happen nearly as much, since over a 30-day period I was only able to retrieve 142 conversations.

Methodology

Looking at all of Cascade’s 14,000-plus Followers and Friends, my goal was to analyze the conversations related to cycling within Cascade’s universe. I attempted to create as broad a search as possible, encompassing not only keywords related to cycling but also to urban policy and road use. Here are the quantities I found:

• Among Followers: 100,203[1] bicyling conversations out of 376,893 total tweets from all of these users.
• Among Friends: 12,750 bicycling conversations out of 118,526 total tweets from all of these users.

Then there’s the other component of this study: the media. One of the reasons, aside from being a cyclist, that I chose to look at cycling in these venues is because it is one of the few topics, such as same-sex marriage and the Middle East, that draws the same high amount of “discussion” in news sources. Therefore, I looked at multiple articles, including: “Seattle’s street parking vanishes as bus and bike lanes boom” (Seattle Times, Feb. 14, 793 comments) and “1 million bike trips this year over Fremont Bridge” (Seattle Times, Dec. 29, 2014, 123 comments), and “Seattle Driver Blocking Bicycle Lane to Cyclist: ‘I Literally Don’t Give a F#$^ About Anything You Say,’” (The Stranger, March 26, 2015, 70 comments). Incidentally, the comments from that article were actually much more bicycle friendly overall. The article I used to create my data sample came from the Seattle Times: “Protected bike lane on Seattle’s Fifth Avenue may be in future,” (May 24, 2015, 352 comments).

In relation to most other articles that run on these sites, cycling-related articles draw high numbers of comments. The “Protected bike lanes” article came closest to the average number of comments added.

So what types of conversations or posts are taking place? In looking through all of the data, I was able to break them into five categories:

• Positive: Congratulatory messages, status updates, days out on the road — anything where cyclists or friends of cyclists had something positive to say about the activity.
• Negative: Anti-cycling messages, bike accidents or bike theft reports — though they are very different types of posts, and I was not seeking to do a study of bike theft in this project, there were enough that I felt it warranted to place them in this section.
• Policy and legislative: Council or legislature meetings on roads, bicycle policy, or activities around bicycle infrastructure.
• Sales: These were primarily bike or accessory discounts from bike shops, with a smattering of Craigslist listings thrown in.
• Events: These covered anything from races to bike festivals that informed people about an event or reported on it if they were participating at the time or afterward.
• In addition, a number of conversations picked up in this data request may have included the search terms but didn’t relate to the subject matter. Tips for recycling materials or tweets related to motorcycling were most common.

Given that I had a dataset of well over 100,000 to work with, I created a representative sample dataset by randomizing the order of the set and pulling:

• 1,000 Follower conversations
• 1,000 Friend conversations
• 142 non-connected conversations (the complete set)

For questions that sought specific information about numbers of retweets, for example, I used the entire dataset and not just the samples. A look at the sample datasets can be found here.

Findings

Here’s how the numbers from Twitter ultimately revealed themselves.

What really sticks out is how much positive comments dominate the Twitter conversations over everything else, and how few negative comments appear in this data. This won’t be the case on the media data.

For a better breakdown of percentages in their respective sections:

Positive Comments

Clearly, among all users, positive comments are in the majority. In looking at the positive comments, in particular among the Friends, however, we can see that these tweets can range anywhere from enjoying a day on the saddle:

Had a great weekend of riding on beautiful Whistler Bike Trails with breathtaking views! #whistler #fatbike #bicycle https://t.co/fkRCDVoqus
— By @surface604, March 31, 2014

To profound (and on point with this entire study):

@turbinewind Angry tweeting does sound kind of silly almost like angry cycling seems kind of absurd. Now you know why I prescribe to wine
— By BikeLifeCities, May 18, 2015

To simple messages or requests:

Forgot your bike in #pdx after the #BikeSTP ? I have it here in #Seattle. Come pick it up so I can go on vacation. https://t.co/8y88mG1YIY
— By heyitsweintraub, July 15, 2014

This was similar straight across the three types of users.

Negative Comments

When people made negative comments, they more often than not were a result of a bike theft or an accident on the road.

RT @SeattleEBike: Cyclist killed on Seattle’s notorious 2nd Avenue bike lane. https://t.co/R9xoXlmEYP @theseattletimes #bike #SEABikes #WABike
— By @PugetHiker, Aug. 29, 2014

And

RT @seabikeblog: Just in: Person killed in Dexter hit-and-run identified as Mike Wang https://bit.ly/nnwTl5 #SEAbikes
— By @BicycleHeroes, July 30, 2011

Unfortunately, both of these cyclists killed were friends of friends. This is a reason I pay attention to these interactions. However, when it came to negativity about cycling itself, the posts were more reactive to negative events than fomenting negativity:

In my work with hundreds of law enforcement, residents and motorists, cyclists who run stop signs/lights is #1 complaint. Easy prob to fix.
— By @cyclist_lawyer, May 31, 2015

The few negative tweets from accounts not connected to Cascade did appear more aggressive than those that were within. Two cases in point:

@TeamRobS 1000% agree with that. also the amount of people i want to shove off their bike going up dexter each morning is TOO DAMN HIGH
— By @themanamedan, May 19, 2015

and

RT @AdiaBarnes: Why was I walking on campus and a 70 year old lady on her bike told me “move you dumb jock”. I heard she is famous for that.
— By @KBierria, May 20, 2015

These types of posts are far different from what you’ll find in the media, as we will see later in this report.

Policy Posts

This is the area where we might hope to see people using Twitter to influence policymakers, as well as see where policymakers might be looking at Twitter as a means to hear what voters want. As you can see in the graphs above, policy tweets numbered a distant second to positive tweets among Friends and Followers, but fell to fourth place among the unconnected. Among these tweets, the conversation centers more around policy itself as opposed to reaching out to policymakers:

RT @altaplanning: The Protected Bike Lane boom continues! Check out the rad new lanes that opened up just yesterday in Seattle: https://t.co/[address cut off]
— By @miabirk, Sept. 9, 2014

However, among Cascade’s Friends, there was much more outreach to elected officials, in particular @MayorEdMurray:

RT @CascadeBicycle: Thank you @SeattleDOT @mayor_ed_murray @skubly! | Interim protected bike lane now open on Roosevelt Way NE https://t.co/[address cut off]
— By @Cascade_Jeff, Jan. 23, 2015

Among those not connected to Cascade, any policy discussion was limited to infrastructure and didn’t call out any elected officials by name.

So how does Twitter affect bicycle policy?

It’s hard to know if the mayor and his staff, or the city council for that matter, are looking at Twitter and using it as a gauge, but if their responses or retweets can be any measure, then the answer is not at all. Elected officials in Seattle (and plenty in Portland and Vancouver as well) are called out either for the work they’re doing or to ask them to get it together on infrastructure issues, but all of the references to these officials are based upon what they’re doing outside of Twitter.

 

One Seattle councilmember, Mike O’Brien, for example, is feted on Twitter for his advocacy work, but he does not respond or retweet that callout:

On stump, @CMMikeOBrien says he’s super-excited to work with Cascade to pass #MoveSeattle Levy! #SEAbikes #SEAelex
— From @CascadeBicycle, March 26, 2015

Tom Rasmussen, who also withheld support from the bike safety plan, tweets more about cycling than the rest of the council. He has plenty of policy stuff:

Delridge ped./bike trail looking north from intersection at Andover. Huge improvement to this route. Thanks SDOT!
By @CityHallTom, May 29, 2015

but he also tweeted recently about Cargo Bikes.

Thinking about a cargo bike? Here are 6 reasons why cargo bikes are the next big thing grist.org/cities/6-reaso… via @grist
By @CityHallTom, Feb. 20, 2015

Twitter use from other council members is limited, if nonexistent. Socialist member Kshama Sawant (@cmkshama) has one mention about some of her staffers who bike to work; Nick Licata (@nicklicata), a huge supporter of pedestrian rights, hasn’t tweeted since 2013, and has nothing about cycling. The best Bruce Harrell gets is a nod from Cascade:

@bruceharrell says tripling bike use is a good thing.
By @Cascade Bicycle, April 1, 2014

Sally Bagshaw, who withheld support from the bicycle safety plan last year, has a good amount of Twitter activity — she actually tweeted a handful of times about bike-related policy:

Everything you ever wanted to know about the Westlake Cycle Track: bagshaw.seattle.gov/2014/11/05/the…
— By @sallybagshaw, Nov. 6, 2014

Tim Burgess (@CouncilmanTim) hasn’t had a tweet about cycling since 2009, while Jean Godden (@Jean_Godden) has a handful of tweets, though none recent. And finally, Sally Clark (@sallyjclark), who retired from the council a couple months ago, has some pictures posted by others of her visiting biking events, but the most recent is a lawsuit against her by a cyclist she hit while driving. As a persuasion tool, it’s clear that Twitter is not a driving force when it comes to bike policy.

Interactions and Conversations

Among these thousands of tweets, a large number are interactions and conversations, i.e., retweets and @ conversations directed at others. I had two questions I wanted Twitter to answer within these conversations:

1. Does Twitter affect policy in municipalities when it comes to cycling? At least in the City of Seattle, as I noted above, the answer is a definitive no.
2. Do all of the various cycling organizations within the region have their own echo chamber when it comes to sending out their messages, or do those connections go beyond?
3. Do these smaller interactions say anything definitive about the larger bike culture on Twitter?

The answers here are a bit more complicated.

Here’s a look at how those interactions take place:

Followers

If we look at the Followers as a whole, we can see that we have a number of retweets relative to the total tweets on any given subject, and an even smaller number of conversations started. In other words, of the total tweets, only a handful of messages — 35 in the full dataset — have five or more retweets or any kind of back and forth to share others’ messages.

If we break these conversations down a bit further, however, we’ll see some trends in how they’re used:

What we can see is that with the exception of sales tweets, the breakdown is almost even between retweets and conversations. Again, positivity takes a strong lead, with policy coming in a distant second.

Friends

Among Cascade’s Friends (again, those that Cascade follows), the blocks look similar although the @ conversations are much fewer as compared to the retweets than with the Followers. Percentages of those conversations look much different than with the Followers, however.

What does this tell us? Not a whole lot, except that we can deduce that Cascade has some nice friends, but the people it chooses don’t have much to say about the organization’s legislative mission when they start a conversation. They don’t, however, have as much of a problem passing the message on.

But what are those messages, and are a lot of those messages being passed around being heard? What’s interesting about the interactions happening in this area is that very few of these conversations follow any trendline or events, and when multiple retweets occur it’s often from users who are entrenched in the cycling scene.

Though, as I noted above, about 35 messages had five or more retweets, the one that had the most was less about cycling and more about the Seahawks and the bike that player Michael Bennett rode around the stadium following the NFC championship victory.

RT @SeattlePD: Just bring the bike back before #SuperbOwlXLIX @mosesbread72 🙂 https://t.co/Pu0IwOXm85

But that’s clearly an anomaly. Aside from that display of Seahawks pride, the breadth of retweets that did occur among Cascade’s sphere of influence was wide if not varied. For example, the pithy:

RT @sierraclub: Happy #biketoworkday2015! https://t.co/KzMsCwES1r

received six retweets within Cascade’s users — and five of those users were related to cycling based on their handles alone (BikeBlogSC, bkindtocyclists, bikeblognyc, bikechicago, BikeLightFairy).

RT @seabikeblog: This is not an artist’s rendition in a high level city plan. This is downtown Seattle today. #SEAbikes https://t.co/WMxLUEg

likewise, had a clear policy bent, and all five of the retweeters were by name related to cycling. Incidentally, the two highest accounts to be retweeted in the full dataset (not just the sample), @seabikeblog and @wabikes, often find themselves being retweeted and shared by the same users. For example, you’ll see multiple posts retweeted by @barbchamberlain. While her name doesn’t suggest any relation to the cycling community, a quick lookup shows that her profile lists her as executive director of Washington Bikes, a statewide advocacy organization that receives some funding from the biking license plates.

Further, if you look at more the names of the accounts that users are retweeting —@peopleforbikes, @GreenLaneProj, @NoSpandexReq — as well as the accounts of those users themselves — @betterbikeshare, @derobikeracks, @BikeLifeCities —it appears that many of these accounts really are just chatting with each other. A handful of individual names without anything to do with cycling in their handles occasionally appears, but they are in the minority.

Further evidence comes from the unconnected tweet dataset: The fact that there are so few tweets to begin with suggests that people who are connected to Cascade really aren’t having conversations about cycling outside of Cascade.

Speaking of the unconnected tweets, the conversations there happened much differently.

Nearly a quarter of those messages in the positive slot were retweets, which makes it look like there’s a lot of activity, but as can be seen below, you’ll understand why. The breakdown by percentage shows here that for both the retweets and conversations, the positive was far, far higher than any other categories, but negative and policy tweets were much more even with each other.

It appears that with these conversations, however, the people retweeting were likely more interested in the people being referenced as opposed to the act of cycling itself. In fact, of the 38 retweets in this sample, 30 of them were based on these two:

RT @JohnHopperstad: #Seahawks player Michael Bennett back on police bike for Bike to Work Day w/ @MayorEdMurray #Q13FOX https://t.co/xITSNss

And

RT @unrulyandrews: Riding bus home from FURY ROAD on the bike rack doing air guitar.

Seahawk Michael Bennett makes an appearance here once again. Most of the rest, however, were about cycling as an activity.

Looking at the News

So how do these results compare to what happens in comment sections on news sites? For one thing, they’re a lot less positive. For another, many of these commenters — like nearly half — have trouble staying on topic.

In many ways, the comments section is a different beast from Twitter. Readers or viewers of articles are, theoretically, responding to the article at hand as opposed to writing or responding to tweets that appear to come out of the ether. Depending upon your perspective, people will have “conversations” or “discussions” or “flame wars” in these sections, and far more often than they do on Twitter. It certainly appears as if people are talking at each other rather than to each other. A likely reason for this is because the article is a self-contained forum, where Twitter is not, and therefore easier to track in one place. But more often than not it’s a place for people to state their opinions, whether or not the facts comply.

In the article about bike lanes (“Protected bike lane on Seattle’s Fifth Avenue may be in future,” May 24, 2015), you can easily see how discussions can nest themselves deeper and deeper — in this example, the negativity breeds further negativity.

Unlike in the Twittersphere, however, where most negative comments centered around such topics as disparate as bike theft and cyclists injured on the road, the vast majority of these comments were negative toward:

• Cyclists’ behaviors
• Transportation policy around bike lanes and facilities
• Elected officials who promote cycling.

What breeds that negativity is more a sociological question than a data-focused question, but in recent years many media companies have resorted to third-party services or Facebook to better manage users making comments. A common complaint had been that people were more likely to be positive if they were forced to show their faces, so to speak. But like a standard comment forum, Twitter users can tweet with relative anonymity if they so choose, so that argument doesn’t hold water here.

The majority of “unrelated” comments could be considered either “troll bait” or negative comments toward other commenters.

Limitations and threats

Overall, I believe the data I collected is sound and useful. Because I randomized the sample data, but used the full dataset to compare the retweets and conversations, I believe I was able to get a full spectrum of data with many of those tweets based in the Pacific Northwest. Because of limitations in the Twitter API related to search, however, I did retrieve Friend and Follower conversations that took place worldwide and without a specific date range. But all of these users were connected to Cascade, which is a fiercely local organization and has a better mix of Followers than organizations such as Washington Bikes (@wabikes), with a combined Friend/Follower count of about 8,000, or Seattle Bike Blog (@seabikeblog) with a nearly identical combined user base.

At the same time, to search for users outside of Cascade’s sphere, I was able to do a date-based, geo-located search, which I limited to 25 miles from downtown Seattle. Based upon Twitter feeds in the past month, only 142 Twitter conversations that occurred were related to cycling. It’s entirely possible the dataset was too small, but if that’s how many people are tweeting about cycling then it’s reasonable to assume that expanding the base would bring in more conversations, but they would be less relevant to the data I wanted to collect.

In addition, I could not collect Friends/Followers and unconnected users the same way, so it’s possible that using different methodologies in the data collection yielded different datasets. But given what was available, it appears that all of it was useful and usable for the purposes of this study.

Classification of tweets could prove problematic. Though the five categories (plus unrelated) I created fit most tweets in the dataset, a handful were either ambiguous or a convincing argument may need to be made that they would fit in any, so the placement of some could be considered arbitrary. Overall, however, the way I divided these categories fit the vast majority of the dataset.

One anomaly that could skew data is in the negative comments. Because negative comments encompassed a broader range in the Twitter data — theft and accidents in addition to angry speech — it’s not a direct comparison. However, the angry or anti-cycling negative comments were so small in the Twitter datasets as to be practically non-existent, I don’t see that it made that much of a difference categorizing the other messages this way.

What didn’t get measured? This is where I see the biggest threat to this study. Because this study only looked for users connected to @CascadeBicycle, retweets created by users not related to Cascade would not have been included. So a tweet like:

RT @tooledesign: Because sometimes u just need more protection: heres 2 our “new” #protectedbikelane design #Mondayfunnies https://t.co/RkCu

which received a healthy six retweets among my dataset, hardly scratches the surface of the 135 retweets it has gotten since it was first tweeted on June 1. The original tweeter, @tooledesign, has a combined total of about 2,100 Friends and Followers.

In addition, this study does not compare the users to each other. Because the profiles of Twitter users and the access allowed to commenters is limited both in what’s collected and what’s actually available outside of each organization’s walls, I have no way of knowing if “Make sense please” or “jazzman21” are curmudgeonly old white men or if @BikePathEnds or @Cascade_Briana are women in their 30s (though I can probably safely assume that @Cascade_Briana is Briana Orr, Cascade’s communications specialist, fits that profile). But if I don’t know whether I’m comparing apples to apples, that could create skews in the data.

So let’s restate the question

In an age of social media, do comments sections still have value? If we’re talking from a data perspective, then the answer is a definitive yes. Clearly the comments sections of media sites serve a vastly different purpose and attract different types of users than Twitter.

The conversations happening in the comments sections appear to be “in the moment,” whereas Twitter posts are more likely to be related to something happening in real time as opposed to responding to an article released from a media outlet.

Whether comments sections are good for the future of humanity? That, I suppose, is another story.