Introductions: Why This Report

by Wendy F Hsu, ACLS Public Fellow

In September 2013, the City of Los Angeles Department of Cultural Affairs brought me on board as an ACLS Public Fellow to develop the department’s digital initiatives. Among the digital initiatives that department was embarking on was the social media pilot program. This purpose of the pilot program was to engage with a variety of social media engagement practices and evaluate their success relative to our department’s goals.

My efforts began by researching the best practices in the field. In this emerging body of knowledge, I found, some have made broad inquiries about how citizens and civic organizations have used digital technologies to mediate civic engagements and practices (Gordon, et al 2013). Others have raised questions about the potential benefits of using social media for governments (Landsbergen 2010). Informal and popular publications such as white papers and blog posts have also contributed to pragmatic discourse to provide pointers for governments to deploy social media (Hootsuite, “Social Media in Government” and “Guide: Social Media Metrics for Government”). This how-to literature is mostly geared toward branding and marketing. Because the origin of social media is rooted in the private sector, most of the ideas that go into the design and practice of social media are already structured by private interests such as product awareness and customer service. Even the way social media analytics is modeled — the metrics, approach, and questions raised — is a reflection of these private-sector goals and principles.

To respond to this, I began to interrogate the utility of the existing social media primers. In particular, I ask how we in government can critically adapt private sector tools, approaches, and values for a public interest. I do so while being mindful of our mission as a municipal agency that serves 4 million residents and 40 million visitors, a level of governance with missions that vary so much depending on the local needs of the city. For that reason, a reflective documentation of best practices can potentially help inform other local-level institutions such as nonprofit organizations and other municipal agencies.

My answer begins with DCA Social Report, a report that documents and reflects on the department’s social media pilot project. In 2012, our Public Art Division started a Facebook page to share news related to public art launch events, artist opportunities (request for proposals), public meetings, etc. In November 2013, I set up a Twitter and Instagram account. As a department, we have experimented with various social media engagement strategies via live tweeting workshops, hashtagged campaigns, Twitter parties, etc. It’s time to take stock of what all we have achieved and learned from our results and data.

Through an iterative process of doing and reflecting, we have thought deeply about these questions: What are our goals and objectives for using social media as a municipal government agency? How do we as DCA define our goals and objectives for using social media? How do we accomplish these goals considering our role within the city government, organizational culture, and low-resource constraints? How do we adapt industry and federal-level assessment tools and methods to further our goals? What do we measure? How do we measure?

This report evaluates the effectiveness of our social media endeavors based on six objectives. These objectives were evoked first at a workshop with the department’s senior staff that I conducted. Combining workshop results with discussions with DCA’s General Manager Danielle Brazell and Marketing Director Will Capterton-Montoya, I identified six major objectives and the definition of each objective. These department-wide objectives include: Access, Participation, Relevance, Responsiveness, Transparency, and Visibility.

The structure of this report microsite reflects these six objectives. On each page, you can read how we define each objective and see how we evaluate the effectiveness of that particular objective using our data. The performance results are followed by recommendations for fine-tuning our social media strategies.

How the research was done, what tools we used

I have taken a practice-based approach to implement social media at DCA. I created a multi-step plan that begins with the piloting the program, experimenting with various social media engagement strategies. Then via anecdotal reflections and thorough evaluative analysis, I worked with our digital media intern Jack Moreau to assess the impact and performance outcomes of our pilot efforts. Based on this assessment report, we will adjust our engagement strategies, and ultimately come up with a comprehensive social media plan for the department. This iterative process ensures that our plan is informed by practice and that its task-level specificity is designed with attention to the user experience of both the civic staffers managing the accounts and ultimately the end-users, the citizens that our department serves.

This report is beta for two reasons. We welcome feedback on the content and form of this microsite as we experiment with a new information-sharing paradigm for social media evaluation. Additionally, we consider the purpose of analysis exploratory, not conclusive. This is the first of our efforts in closely examining our social media data. The figures generated in analysis will provide a future reference to create benchmarks suitable for our organization and to refine metrics for future assessments.

To keep track of the performance of our pilot program, we used the native analytics from Facebook Insights. With Twitter, we tracked our data using social data analytics/aggregator Sumall, leveraging all of Sumall’s free-of-charge features. Additionally, we gathered qualitative data related to individual campaigns and the records from each endeavor, ex) the Storify of our grants workshop; a Tagboard of posts related to our LA Islam Arts Initiative (#LAIslamArts).

The work of identifying useful metrics is critical. We referred to DigitalGov’s primer “Social Media Metrics for Federal Agencies” (Herman 2013) as a start and culled through the list to find metrics relevant to our objectives. We quickly realized that off-the-shelf social data aggregators do not suffice the need of information for government-based inquiries. For instance, Sumall and Facebook Insights are good at capturing data related to community size/growth and engagement volume. These basic-level analytics may work well for a company that is selling a product, but it may not capture the experience of citizens’ engagement with government. The metric category of customer experience is set up to measure the satisfaction levels of user experience of government information. The purpose of government information is to enable and educate constituents. These goals fit the market-driven framework from the private sector. Private sector metrics would not be sufficient to address access, transparency, and visibility, all are key criteria for the public mission of government. Additionally, the rates of conversion such as click-through data for the depth metric barely scratch the surface to meeting our goals of public service. A creative integration of analog data such a physical attendance of an arts event or a workshop; or the reduction of informational phone calls could better measure the effectiveness of our communications. The DigitalGov metric primer, however, does provide a helpful index of data that we could refer to as we make plans to collect data for the future (for instance, traffic between website and social media channels) in order to refine our assessment instrument.

To drive the assessment design, I asked the following questions:

Success: What are the successes of our social media pilot? What may have caused these successes? What do these successes mean to us as an agency?

Weakness: What are the areas of weakness in our social media pilot? What may have caused the weaknesses in our efforts? What can we do to improve? What do these weaknesses mean to us as an agency?

Existing data: How useful are the data that we currently have? Are there any data that were not documented this pilot year? How can we improve our data collection mechanism?

Strategy: Are there any areas of engagement that were left out in this pilot year? Where do they fall in our agency’s mission? Should they be prioritized for the coming year?

The design of the analytical approach for this project required inventiveness. We saw the limitations of tools of social media analytics mostly designed for private-sector needs. We applied a mix between qualitative and quantitative analytical approaches. Taking a cross-platform approach, we examined the data while building up an assessment model that would speak to the purposes of government and the specific needs of DCA.

First we took an inventory of all the data we have gathered and compiled a spreadsheet. Applying a close interpretation approach, we typologized post content which allows us to speculate preliminary ideas about the categories of data we have relative to the types of social media practices (e.g. one-off vs. sustained engagement; explanatory vs. descriptive; advocacy vs. informational) that we have engaged with in the pilot period. We also typologized the responses we received through @mention on Twitter into finer categories such as: shoutout, inquiry, suggestion, support, criticism, accolade, discussion. Next, we applied distant analysis to the categories of data that we have established from the previous exercise. To articulate underlying quantitative patterns in our data, we used a combination visualization and mapping tools such as Tableau Public, ArcGIS Online while repurposing charts in Sumall and Facebook.

Data and analytics can be seductive. They can lead us down rabbit holes and unproductive paths. To stay focused, I established the following six analytic principles:

1. Data is never the whole picture. Interactions over social media are rich in meaning and they could be interpreted differently depending on the individual or organization’s perspective.

2. Be suspicious of numbers and what graphs can tell us. The design of charts and graphs can be driven by interests and ideologies of the creator. Always consider how the numbers and data are expressed when drawing conclusions from quantitative information.

3. Think about the scope of assessment. Are you measuring the impact of a particular social media campaign or an ongoing, sustained social media practice? The answer should determine the design of the assessment. A campaign assessment design might include combinations of metrics, based on the desired outcomes of the activity and characteristics such as community size, engagement levels, and responsiveness.

4. More is not always better. Approaching the qualitative impact of interactions over social media takes creative thinking that links public agency’s objectives and indicators.

5. Contextualize expressions of quantitative data with a clear picture of the research question and a narrative about the source of data and method of collection and analysis.

6. Take a cross-metrics approach whenever appropriate. Sometimes the impact of social media engagement can be better measured by observations of engagements over physical, analog, and other digital communication platforms. For instance, one of the best ways of measuring the effectiveness of a digital (social) media information campaign is to see how many/little phone calls we get, or the number of attendants at a live cultural events.

How this report is useful for others

When I started my work as an Public Fellow with the City of Los Angeles Department of Cultural Affairs, one of my first responsibilities was to come up with a social media plan for the department. My doctoral training consisted of digital humanities methodology, using social media analytics to understand the digital diaspora created by Asian American rock musicians. My training enabled me to scrutinize the meanings and formations of digital social relations. Still, the task of creating a social media plan seemed daunting especially after I realized that social media is still a new terrain for governments. Information such as guidelines and best practices of government social media practices is sparse and dispersed on and off line. Because government mission can differ vastly depending on which level — federal, state, municipal — and types and purposes of public service range tremendously, coming up with a template for social media practices, either in the format of a plan or a set of policies and guidelines for all government types can be an impossible task.

DCA Social Report is licensed under Creative Commons Attribution Non-Commercial License. Sharing the recommendations and other outcomes of this report, I intend to contribute to the collective understanding of social media in government. I also hope that this report will spark remixes, adaptions, and conversations that build on this research. Research on best practices and models for municipal governments can increase civic literacy and public knowledge about public-sector communications. Restricting this document in the non-commercial sphere is my effort to steward this body of knowledge within the public realm, thus contributing to the public mission of government.

In the months to come, and by the end of my fellowship, I will produce and publish a social media kit that is based on the iterative documentation, assessment, and engagement endeavors at DCA. To reframe the compliance-based language common in municipal governments, the social media kit will be an enabling tool to encourage generative civic communications and engagement.

Works Cited

Gordon, Eric, Jesse Baldwin-Philippi, and Martina Balestra. 2013. “Why We Engage: How Theories of Human Behavior Contribute to Our Understanding of Civic Engagement in a Digital Era,” The Berkman Center for Internet & Society, The Social Science Research Network Electronic Paper Collection.

Herman, Justin. 2013. “Social Media Metrics for Federal Agencies,” DigitalGov, retrieved from http://www.digitalgov.gov/2013/04/19/social-media-metrics-for-federal-agencies/ (accessed on January 8, 2015).

Landsbergen, David. 2010. “Social Media in Government: Government As Part of the Revolution,” Electronic Journal of e-Government, Volume 8 Issue 2, 2010, pp.135-147.

Hootsuite. “Social Media Metrics for Government.”

Macy, Beverly and Hootsuite. “Social Media in Government”