By David Manuel Torres | May 13, 2019
Research Assistant, ETR
For most people, “data” is a four-letter word—not many enjoy looking at distributions and comparing percentages. Even fewer like to do so at the end of an hour-long meeting on a Tuesday morning. It’s data. If you’re not into statistics, it can be confusing. It isn’t always clear how it’s going to be useful in the real world.
Still, there are times when data need to be discussed. It can be an incredibly powerful tool for researchers and organizations. I’d like to share how we approached the gathering and sharing of data in one of our partnerships in ways that enhanced “non data-oriented” staff’s competence and confidence in understanding data while also informing how ETR does research.
ETR's Equity in STEM team has submitted a video to the 2019 STEM for All Video Showcase sponsored by the National Science Foundation. We share our experiences hosting the first-ever Spanish Family Code Night with local schools and community partners. Please visit the site and take a look at video presentations from a wide range of federally funded projects that aim to improve STEM and computer science education. Be sure to watch our video here. We'd love to earn your vote of approval. (Votes must be recorded by May 20, 2019.)
ETR has been working with staff from the Digital NEST and its predecessor projects for over a decade. This nonprofit community-based technology center was founded and is located in the heart of Watsonville, a small city the center of a large rural, agricultural community in California. The Digital NEST’s mission is to provide technology access, positive youth development, career exploration, job training and paid professional on-the-job capstone training to predominantly Latinx youth ages 14-24 from working-class families with immigrant histories. In 2015, ETR began a research-practice partnership (RPP) with Digital NEST. Our goal was to build systems and effective practices to support technical education and career pathways for underrepresented youth, with ETR as the researchers and the NEST as the practitioners.
The William T. Grant Foundation has an excellent site explaining RPPs and offering resources to support implementation. They define RPPs as “long-term, mutually beneficial collaborations [between researchers and practitioners] that promote the production and use of rigorous research aimed at solving problems of practice.” Their dual goals are to improve the relevance of research and increase its application and utility in solving problems of practice—in this case, with an organization seeking to improve outcomes for youth.
And thus began the long process known as data collection. For over a year, we worked with NEST staff to distribute and collect surveys, track participants, and conduct interviews. Over time, the dataset grew large enough that some exploratory analyses could be performed. We knew the data was going to be more useful to practitioners if they didn’t need to wait three years to use it!
We started out meeting with staff to review some of the raw output of the statistical analyses we ran. But we wanted to offer our material in a more useful format. While the staff at the NEST are sharp and skilled, we are all too familiar with how alien output from statistical software can sometimes look. We opted to create “data briefs,” short, graphic representations of the results organized in a quasi-narrative form. These data briefs made the results easier to interpret.
We shared these data briefs at the NEST’s weekly staff meetings. Soon, we started noticing some interesting changes in the RPP dynamic. When presenting the data briefs, we would walk staff through each of the “sections” (e.g., demographics, pre-data, post-data). We’d note the results that we found interesting, then give staff some time to look over the brief and formulate comments/questions for a group discussion.
In our original presentations where we shared our raw output, this group discussion would be dominated by just a few of the staff members (typically the program leads). However, after switching to data briefs and creating space for staff to talk to each other, we noticed that more people began to participate in this group discussion. From the front desk receptionist to new employees, staff were not only commenting on results, but commenting on and building off each other’s remarks. Some even began brainstorming ways to change their recruitment methods to reach youth who were not well-represented in the dataset, such as women and older youth.
We discovered that these more open and engaging conversations were useful for us as researchers as well. It helped us ask better questions of our data and facilitated the building of trust and collaboration between NEST and ETR staff. ETR was no longer seen as an “outside organization,” but rather as a partner.
We noticed that NEST staff became more comfortable in reaching out to us for support in their various projects. We had one staff member ask us to look over a survey she was planning on administering to members of her “Web + Information Technology” track. We had another ask for help interpreting data she had collected on some of the older, more experienced members. We were even asked to run a short “Research Methods 101” workshop to do a deeper dive into the types of data one can collect and methods one can employ to collect them.
All said, the “partnership” part of the RPP has grown stronger over time. We now have more buy-in from staff around the importance of collecting, analyzing and interpreting data, and the value that adds to a project, especially in the early stages of development. More importantly, we have learned that forming and maintaining an RPP requires that all stakeholders be flexible and understanding of everyone’s background. We plan to take these lessons learned and use them as a springboard in our next collaboration, “A Coordinated, Cross-Institutional Career and Technical Education Cybersecurity Pathway,” where we will be introducing two other organizations into our RPP.
RPPs are a valuable tool for collecting data that has real-world relevance. We hope that more organizations will form these types of partnerships. When researchers use their knowledge of conducting research and looking at data objectively, and join it with the wealth of cultural and practical knowledge that practitioners bring to the table, we get something that is greater than the sum of its parts.
Photo credits: Digital NEST.
David Manuel Torres is a Research Assistant at ETR. He studied Cognitive Science at the University of California, Santa Cruz and learned during his studies that he enjoys working with data. He can be reached at email@example.com.