OSU worked with the state to build the OLDA to help increase access to administrative data for research purposes. 2005. Ohios application for first round of the Workforce Data Quality Initiative was submitted in August 2010, leading to six years of direct funding from the USDOL for Ohio to build a state longitudinal data system.

Hawley, Joshua D., Dixie Sommers, and Edwin Melndez. OSUs legal staff worked directly with the Ohio Attorney Generals Office.

Since approximately 2009 the centers have partnered with the State of Ohio to store Ohio administrative records. While there are no formal differences in application, there are some dissimilarities in terms of the kinds of data that government requests and the projects they propose. Jacob France Institute, University of Baltimore.

There are a relatively small number of research centers with state administrative data in the United States. The adopted safe person rule has a requirement if a student applies: it is only under the supervision of a researcher and that they understand the extensive time it might require to wait for data access. The WSM includes information on all of the programs included under the federal Workforce Innovation and Opportunity Act. The list is maintained in a bibliography211 and is not inclusive of in-progress work or work that has been submitted to the center but not yet reviewed or finalized. This is easy to accomplish as the files are simply combinations of existing microdata. Outreach was also supported by presentations at national meetings, such as the National Center for Education Statistics (NCES) STATS-DC Data Summit and Workforce Data Quality Initiative convening in Washington, DC. Education data In the case of Florida, Illinois, Maryland, Texas, and a handful of other states, the data systems have existed for at least as long as Ohios in some form or fashion. Making Good on the Apprenticeship Promise Will Require Major Investments in States. Ohio Gadfly Daily. New Growth Group. What seems important is that government must see data as a resource to improve outcomes as opposed to something to limit access to. The Ohio Longitudinal Data Archive (OLDA) is a collaborative arrangement between the State of Ohio and the Ohio State University (OSU). Projects can be rejected at this point, a stage akin to a desk reject from a journal. The Workforce Data Quality Initiative provided funding for the establishment of the OLDA longitudinal data system in Ohio. The OLDA maintains data from five different state agencies.

The university conducted research on an ad hoc basis between 1995 and 2010. Ohios Adult Workforce Education System: An Initial Investigation Using Administrative Data. Report. Finally, there is an openness to university life that enables more innovation with data science. The office location of the computer at the place of work is collected and it is required that it is a desktop, not a laptop computer. This is necessary to ensure that any findings or results from the study are communicated to the data owners prior to being published or presented to other groups. A second major reason for expanding data systems was an increasing demand from researchers for unit record data. Borus, Michael E. 1982. With this system researchers get access to a range of information on the relevant data. Researcher extracts are maintained permanently on OLDA systems. That original proposal was a collaborative effort between the Ohio Department of Job and Family Services, the Ohio Board of Regents, and OSU. The specific files maintained at the Ohio State University are detailed in Table 8.1. The proposal declared that the team would aid the State of Ohio to incorporate workforce information into longitudinal data systems, to help follow individuals through school, into and through their work life.206. Both the Code of Federal Regulations 20 (Section 603) and the final regulations of the WIOA (Final Rule) are necessary in legal agreements when wage records and job training data are to be used. Safety in terms of personnel is ensured in a number of specific ways. https://etd.ohiolink.edu/!etd.send_file?accession=osu1366224121&disposition=inline. Currently, there is a multi-stage review mechanism in place that screens safe and unsafe projects. Depending on the data file, staff make some changes to ensure that the data do not identify high-earners or people enrolled in very small enrollment programs.

WIOA makes it clear that states are required to participate in evaluations to the extent possible. The initial idea for the data system emerged in 2007 out of a partnership between faculty at the university, which resulted in an MOU giving OSU access to state data.202 The OLDA is a linked to a college research center, the Ohio Education Research Center (OERC).203 The OERC is a policy research and evaluation unit at the Glenn College and conducts contract research with state and local government. Individuals submit these queries for data extracts, and these data dictionaries are maintained in individual user accounts as well as by the research team. 2017. https://doi.org/10.1002/cc.10402.

Hawley, Joshua D., Tian Lou, Randall J. Olsen, and Christopher Spence. 2017)., The Ohio Revised Code section on the Education Management Information System describes the system and its legal basis (ORC, Chapter 3301-14)., Ohio Department of Job and Family Services (2010). Changes in personnel, such as the addition of a research assistant, must be negotiated ahead of time. State and local government offices are often asked to participate in long-term research in collaboration with the federal government. At the creation of the formal data center, OLDA staff conducted a range of outreach activities to socialize educational administrators in Ohio and external researchers. Joshua D. Hawley is a professor in the John Glenn College of Public Affairs at the Ohio State University (OSU). Researchers obtain the Ohio data by completing a standardized set of documents and obtaining Institutional Review Board (IRB) approval. Office of the Federal Register. Federal requirements for evaluations, particularly in the Department of Labor, are important reference points for legal and program officers in state agencies, as the federal laws allow for use of data to evaluate a public program (Code of Federal Regulations 2006).

Individual files that generate statistical results for publications are maintained by approved researchers.

Report on the Ohio Closed Cases Study. Report. First, almost all doctoral granting institutions in the State of Ohio were collaborators in the original proposal.207 Second, Race to the Top required a prodigious number of independent research and evaluation studies that made use of administrative data between 2012 and 2017.

There are two models of administrative data centers, States such as Kentucky maintain an administrative data center inside state government.

Projects must be related to either policy or research priorities of state and local government.

The OLDA project must comply with the typical rules for academic research projects.

There is no option to submit an application for data without this letter being available ahead of time, unlike with National Science Foundation (NSF) or National Institute of Health (NIH) proposals. The legislative process, including the biennial operating and capital budgets in Ohio, creates regular demand for research using state data. The team has built several that are maintained every year for over five years and some that are more recent. https://ohiotechnet.org/wp-content/uploads/2018/10/Ohio-TechNet-Evaluation-Final-Report.pdf. Researchers at other ADARE institutions, including the Upjohn Institute and the University of Missouri, were very important resources for each other (Stevens 2012). Operated jointly by the John Glenn College of Public Affairs and the Center for Human Resource Research (CHRR), the OLDA stores data from five agencies (Education, Higher Education, Housing, Job and Family Services, and Opportunities for Ohioans with Disabilities) in Ohio. The OLDA is an example of long-term partnerships between state government and research communities. Governors, the legislature, technical staff in the executive branch, and the additional stakeholdersincluding academic communitieswork within a common political environment. In Ohio, researchers have been lucky to have an operation that goes back in one form or another to 2000 and even further back for some research projects. The Ohio Education Research Center website maintains an archive of research studies conducted under the Race to the Top project. This policy action at the state level was necessary to ensure that school funding was provided under the American Recovery and Reinvestment Act ARRA mechanism.

The Ohio Technical Skills Innovation Network (Ohio TechNet). Report. However, if an agency requires deletion of the data, OLDAs legal MOU requires compliance. Table 8.2 provides example titles from approved research projects. In some states, such as Illinois or Texas, these data are shared with researchers, while in Ohio both federal programs data are largely off-limits to the research community. In 2019, the data holdings came from the following state agencies: Within each agency, the data resources include the core agency-specific files for federal and state administered programs, such as the Workforce Innovation and Opportunity Act (WIOA). The specific data a researcher acquires and then uses depends on the analysis and the questions proposed. The measures used include the number of individuals completing the program, the number of these individuals subsequently employed in Ohio, the median earnings of these individuals, employment stability, college enrollment, and education and training credentials earned. There are five forms that the individual researcher must file to apply for access: (1) a data use procedures and checklist, (2) a confidentiality form, (3) a data use agreement, (4) the Collaborative Institutional Training Initiative (CITI) responsible research use certification document, and (5) an institutional review board (IRB) approval letter. This MOU is initiated by one of the member agencies (Ohio Department of Job and Family Services), signed by all the remaining agencies (Education, Higher Education, Housing Finance, and Opportunities for Ohioans with Disabilities), and thereafter by the Ohio State University. Several of these projects relied on data from across institutions as well as different state agencies.

In both cases the university received data on an ad hoc basis, straining both the technical systems to ensure security for private student records and the legal frameworks in Ohio. A recording of the webinar and slides are available.

First, working within a university setting is somewhat insulated from the day-to-day politics, compared to being embedded in a state agency. For example, each file is documented in a standardized manner in the investigator so that individual researchers can compare the kinds of variables they will receive. Organizational success also requires consistent political support. All researchers sign legal assurances that they will delete the data provided after a certain period of time. Using State Unemployment Insurance Wage-Records to Trace the Subsequent Labor Market Experiences of Vocational Education Program Leavers. Report. This is necessary to state because supplementing the data with additional survey or administrative files might make it possible to identify people.

FERPA prohibits the release of individual student data, barring certain exceptions. https://fordhaminstitute.org/ohio/commentary/making-good-apprenticeship-promise-will-require-major-investment-states. As education organizations (e.g., schools or colleges) in the 1990s moved to using databases to manage regular businesssuch as registration, course enrollment, or testingstate agencies supervising these schools developed the data systems to help schools and universities carry out the day-to-day work. 2012). The core data holdings from the wage records and all public education and higher education providers enable researchers to answer critical questions such as (1) what are the employment outcomes of higher education, (2) what kinds of industries are growing or shrinking, and (3) how does employment depend on major or credential?

The amendments in 1994 allowed the federal and state government to allow access to student data under some conditions.

If a researcher has a year to use the data, they must delete the files at the end of that year.

Secondly, all staff directly working on the OLDA are OSU employees and must adhere to policies, including data security training. A second stage of review for safety is conducted with a more formal application, which allows the research team to make the case to state agencies as to why the project is appropriate.

Determining which projects are appropriate is complex and changes over time. Traditionally the researchers pay attention to cell size and geographic level of disaggregation.

This federal funding was dramatically expanded after the State of Ohio hired OSU to build the Ohio Education Research Center as a deliverable for the Race to the Top Project in 2012. The research uses of the OLDA depend on the programs that contribute microdata. The OLDA is a collaborative project between the State of Ohio and the Ohio State University that is categorized as a funded research project at the university; as such, it operates within a university institution. There is also a search function for variable names and pre-coded topics.

At Ohio State, the team created the technical approach to merging state data, doing probabilistic matching, and standardizing data reporting rules. In fact, these must be deleted at the end of the approved period of time. OLDA presented for approximately six years at these meetings to states, serving to get the word out about state level use of research data to improve programs. Student data access is only allowed under the supervision of a faculty member because data access often takes a year or more.

Individuals begin to analyze data by selecting a data source (e.g., higher education) and subsequently limiting the number of variables and time periods. Individual chapters by their authors or as noted. Harlow, Kristin. This is a workaround for citations in footnotes, please ignore. As the team gained experience, there was a shift from more directed calls for research in specific areas to research on topics that addressed priorities that came directly from researchers without any guidance. https://doi.org/10.1002/sdr.1575. An Effectiveness Assessment and Cost-Benefit Analysis of Registered Apprenticeship in 10 States. Report. Metrics that measure the organizational capacity of the research center are much more difficult to quantify.

The legal basis for these data systems came from Ohio Revised Code.205. https://studentprivacy.ed.gov/resources/integrated-data-systems-and-student-privacy. All projects must be approved by each of the agencies which own data that is requested. At the outset there were frequent meetings between policy level staff, particularly during times of transition in the governors office.

Technically, these files are not replaced over time; additional years or quarters of data are added to the existing file structure. Center on Education; Training for Employment, The Ohio State University.

These two overarching legal documents govern the rules for both government and external researchers.

There was a research advisory committee that included tenured faculty from almost all schools in Ohio.

Individuals complete a one-page project description to ensure that a project is acceptable without requiring technical details on variables. Data access is allowed on the work computer that individuals declare in the application process. The group included a former state director of Labor Market Information, the current (now retired) Labor Market Information director, a deputy chancellor (and former state finance director) and several professors, including the director of the Center for Human Resource Research. For example, individual staff will work for a chamber of commerce, subsequently move to a higher education institution, and might move to an executive role in state government. 2012. It is updated annually. The Center for Human Resource Research team built the dashboard and maintains it.

In later years, the research team broadened the application to topics that could be user identified (projects the State of Ohio had not yet conceived). The metadata are published in an open application on the center website. 2012. A parallel approval process is in place for review of findings.

OSU received the extracted data from the agency data systems from two separate longitudinal data systems independently. 2001.

The Data Stewards are concerned with detailed questions, such as how should significant changes in the definitions of variables be handled. The OLDA data are available through a purpose-built and proprietary software system that is maintained by the Center for Human Resource Research at the Ohio State University. http://chrr.ohio-state.edu/content/surveys/closed_cases/reportpart1.pdf. Initially, there are some similarities across the data uses.

Some of the Ohio agencies actually direct researchers to OLDA systems. Therefore, a description of the data holdings will shift over time as the memorandum of understanding that govern data exchange are altered to meet the policy priorities of government and the needs of specific researchers. Data are also made useable because the research team provides guided support for applicants. For example, Kentucky has a superior legal situation because the state laws formally designate a state office (Kentucky Statistics) as the data system.

The kinds of decisions these committees can make vary, but the Policy Committee is responsible for big questions, such as what data files should be included in the archive. 2003. Access is also limited to specific data elements and research questions. OSU worked with the Department of Job and Family Services on a periodic basis between 1995 and 2010 to conduct studies using wage records in combination with a wide range of other data files, including those from Aid for Families with Dependent Children and Workforce Investment Act programs (Center for Human Resource Research 2001). The entire system is governed by a memorandum of understanding that is renegotiated every two years. The OLDA and OERC are actively used at Ohio State University in teaching education policy, data sciences, and simulation and modeling.204. http://www.jacob-france-institute.org/wp-content/uploads/ADARE-publications-presentations-compendium-11-8-12.pdf.

Both of these activities, legal and technical, help to sharpen ones understanding of the political nature of data-based decision-making in modern government (Stone 2012).

This project exemplifies the ways that a data system can be the foundation for a consistent research project that can assist state and federal government. 2020 Cole, Dhaliwal, Sautmann, Vilhuber. He also serves in leadership roles at two research centers of OSU: director of the Ohio Education Research Center at the Glenn College and Associate Director for the Center for Human Resource Research (CHRR). For example, the Ohio education data began in 2001 and now includes data through 2019.

Ohio has had over ten years of consistent political leadership on data and workforce developing, leading to strong foundations for research work in collaboration with state and local government.

2016. The legal foundation of the current wage record system is based in the Federal Unemployment Tax Act of 1937, which set up a federal tax to cover unemployed workers. A safe project is one that addresses a policy priority that the state is also invested in understanding. The registered apprenticeship work conducted in collaboration with the State of Ohio and the DOL required detailed microdata from RAPIDS as well as the Unemployment Insurance Wage Records and the Quarterly Census on Wages and Employment. OLDA has had occasions where the data for specific projects must be deleted but not the underlying microdata. Evaluation of College Credit Plus: Dual Enrollment in Ohio. Thesis, The Ohio State University. The primary project for the Ohio Analytics Partnership is the Ohio Longitudinal Data Archive.

For example, researchers from Ohio Housing Finance Agency (OHFA) are currently collaborating with researchers at the Ohio State University on an experimental analysis of housing supports on employment.

OLDA does not keep researcher generated files except for those submitted to the disclosure review process. For example, under the terms of the research they must complete OSUs CITI training for human subjects as well as the security policy and confidentiality agreement that is held by the Center. The Ohio State University. There are some lessons from this story that are relevant to other states attempting to build integrated data systems.

FERPA includes an audit and evaluation exception that allows for state or local education authorities to cooperate with an integrated data system (IDS) to access student records to ensure that evaluations of government programs receive the linked data needed (Privacy Technical Assistance Center 2017).

Research output includes 28 published studies in the last five years. The newest one was added in 2017 and the oldest one prior to 1999. Group Model Building Techniques for Rapid Elicitation of Parameter Values, Effect Sizes, and Data Sources. System Dynamics Review 33 (1): 7184. These include academic articles, working papers, and presentations submitted to the research center. The specific language is to provide public benefits. The governing policy council values research that can help understand the impact of priority state policies, such as eliminating social promotion in third grade or reducing infant mortality. In collaboration with the Ohio Department of Job and Family Services, researchers have built a dashboard to compare supply and demand on workers in the state. https://www.govinfo.gov/app/details/FR-2016-08-19. Access is also time limited. The Earnings Impact of Adult Workforce Education in Ohio. In. Since the end of the Race to the Top and Workforce Data Quality Initiative, OLDA has worked on outreach with established research teams as well as responding to individuals directed to our team by agencies. Center for Human Resource Research. As part of the tax, states were asked to build (over time) a way of reporting earnings on a quarterly basis. In the early stages of the OLDA, individual access was limited to studies that were explicitly encouraged under the Race to the Top or Workforce Data Quality Initiative applications.