Episode 8: Education Policy and Problem-Solving with Samuel Workman and Deven Carlson

Hello and welcome to Episode 8 of the Policy Agenda podcast, I’m your host, E.J. Fagan, and today

I’m joined by Katie Matal. Hello. I got your name right, Katie. Yeah. And and on the phone,

I am joined by former project manager of the Policy Agendas Project, Sam Workman,

who and his colleague at the University of Oklahoma. Devon Carlson. Hi. Good to be here.

Yeah, we’re glad to have you guys on. Can you guys tell us a little bit about this project titled

Organizational Problem Solving and the Use of Research and Education. The

larger project stems from sort of an interest in how it

organizations process information. There’s a lot of older literature, especially

Kinne Arrows, Nobel Prize Lectures and

Tom Hammond’s old article about the effects of organization on agenda setting.

And what Šero notes is that the prime importance of organization

is in channeling information while we’re peeling that back to a more base level. And

that is that organization really channels attention to things. And

when you put that together with Tom Hammond’s work on agenda setting, of course,

then organization becomes important for the agenda. As I know both of your advisors would say,

what is the organization? What does organization mean in this sense? Organization

has to do with how bureaucracies are passed up in terms

of the problems they solve. Some Department of Education, for instance, you have

organizations that are dedicated to student populations, organizations dedicated to

Title 9 things. Organizations dedicated to accountability. Devin can speak

a bit more to that and how it maps on to the agenda.

Yeah, so I mean, within the Department of Ed, you have the Office of Post-Secondary Education, the all

sorts of elementary and secondary education. And what we’re what we’re interested in is how these different

organizations work to process, solicit and process information

to address problems in the policy space. And so ultimately

you can think about this as push effects and pull effects of organizations, the

pull effects, the supply of organization, that is, if you’re the office of Accountability,

you’re going to solicit information, be more attentive to information that pertains to the accountability

of school systems or teachers or districts or whatnot. You’re going to then use that information

to push that into policy so that information informs resulting regulatory

policy. So can you give us an idea of where this project is at the moment?

So right now, we are in the midst of coding all of the comments.

The universe of comments on every education proposed education regulation

in about the past decade by paragraph, and that is about 90000

comments. We don’t yet know how many paragraphs that is because we’re

working on passing a lot of p_d_f_ submissions.

But the importance of that from a policy agendas perspective. Anyone interested in agendas

in any manner is that it allows us to get the sort of the the the politicking

that occurs out in the policy community.

So let’s first off, ninety thousand paragraphs are ah is ah, some poor graduate students coding those

or is this. Is this all machine coded? So that’s ninety thousand comments,

each of which as several we’re probably looking at millions of paragraphs,

I would say so. So now some poor graduate students know. So right now we

have a cadre of undergraduates who are passing each of those comments into

paragraphs and then a second group of undergraduates at that with being directed by our

graduate students and Sam and myself, who are doing the substantive coding, who are coding it by

granular policy topic, as well as for the use of research evidence, the presentation of data

within that within each paragraph in sort of this this is mostly sort

of devins expertise on substantive education policy. What we did is we took

the policy agendas, project codes and broke them apart, wrote the examples. They always

list almost things apart and added to a bit in order to study

a more specific policy subsystem. So I actually have a question

for Devin, I believe. Why? Why did you choose the last decade?

So you say in your proposal that you focus on the post Every

Student Succeeds Act y y the past decade. Why not a different decade in

education history? That was partially dictated by data reliability

regulations, dot gov makes easily available comments from

the past decade, but prior to that it’s a much heavier lift to go find those, attract those down.

So one of the easiest places we’re kind of switching back and forth between expertise

here, but one of the easiest places to get informational regulatory commenting

is regulations dot gov. A couple of things to know about it. It it doesn’t go

back much. Ask that, at least for the department of it. And the other thing to know is agencies submit

to it voluntarily, not by law. And so we’re kind of Battleland about it.

The other tack you can take is of course to fully in the comments and they send

you boxes and reams of paper that you can transcribe or something like. Sam,

can you give our listeners a brief overview of how the notice and comment period works and

what you’re aptera actually observing in these comments? Yeah. So so what happens to

the notice and comment procedure is outlined in the Administrative Procedures Act nineteen

forty six. And what it says is that when bureaucracies want to make policy,

because that’s what we’re actually talking about here is bureaucracies making policy, they issue a regulatory

proposal once they issue that Kozel, it’s published in the Unified Agenda and

in the Federal Register and that initiates it’s a proposed

rule that going or rulemaking and then initiates a comment period

where any any group, any person in the United States, even,

for instance, convicted felons, can comment on federal regulations.

Once the agency receives those comments, it compiles them into sort of a summary

or understanding of the major points of the comments or key information. And

by law, and as dictated by the EPA, agencies

must respond to their to those comments in the revision of the regulation or be able to cite

a legal or problem informational reason

why they did not do so. That’s defensible in federal court. And then that is used

to inform when we’re informing these regulations that the implication that there’s an impact

by the comments on the actual law. It’s not just proforma. It’s

it’s not just performace. In fact, it’s legally not just performer.

Now, you’ve touched on this earlier, but can you explain in more detail the theoretical

relationship you expect somewhere in which direction does information flow and

why does information flow in that direction? Well, the key

point here is, is really the impact organization on this information.

And so a lot of interest groups, studies of regulation and

Devon could speak more to this within education policy. But a lot of the studies of interest groups sort of

assume that interest groups have this agenda. They provide information and then they sort

of force bureaucracies to do things by pushing levers. What we’re saying is

that because bureaucracies are not just reactive

institutions, that they make policy, that their organization conditions the supply of information

from the beginning. Interest groups can only react to regulations that are proposed.

Right. And so the bureaucracy sets the agenda in that regard

and how the department structures their their proposed rule. The topics on which they propose rules.

That is all going to structure the information that the department takes in from interest groups

and and other interested entities. And you can imagine a situation in the Department

of Education, for instance, where you have Title 9 challenges or

civil rights and liberties concerns and concerns about racial disparities

or class disparities. Right. Those organizations may very well

be more amenable to legal information than sort of policy research. Whereas

if we get into an area concerned with teacher quality or student outcomes,

that’s that’s organization may care more about the policy research than legal

requirements for fairness and equality in these sorts of things. I mean, I mean, is stakeholder impact

a big part of this to. I mean, we yes, we are interested in looking

at the different groups that come in. Who comments on what topics, how do they present information?

What type of information do they present? How is that considered in the regulation? And so we’re looking at the

interplay between the organizations within the bureaucracy, within US Department of Ed and

the stakeholder groups on the outside that provide the information. And so we’re really looking

at that dynamic rather than just each entity on its own. It’s not

a simple story of saying that bureaucracies dictate policy or interest groups dictate

policy. This is all about the uptake of information, the creation

of demand for it, and how it gets process. Looking on organizational structures

and rulemaking is uniquely is uniquely a uniquely good

place to look at this because groups are free to participate or not.

It’s not like a hearing where they have to be invited. And so we can see who chooses to participate

versus who sits this out on both the presentation of information and the topics

on which they choose to come. How generalizable is all of this? I mean,

you’re focusing on the one case study of the Department of Ed, but how generalizable to who is this to other

departments and other policy areas? Well, it just so happens

that I have. Contemplated this across multiple policy areas

for a good portion of my career at this point, and I would say that this is very

generalizable and it’s generalizable for some specific reasons related

regulations. Regulations aren’t like hearings. They’re very detailed.

The sort of text well, we’re seeing text, but the sort of arguments

and descriptions are very tightly tied to the nature of the problem.

And so you don’t get the Save America Act or anything being issued by bureaucracy.

That means that you’re able to study policy specific

policies and and sort of examine specific argumentation

attached to those policies in a way that you simply aren’t in nearly any other branch of

government. I like to talk about education policy more specifically now.

What during this period, what problems and policy areas, I guess generally are you

are you observing? What’s the what’s the most common policy discussion going

on at the department education? Well, I think there is. You know, given

the breadth of the Department of Ed, there is a few different policies that were being discussed. There’s that there’s

a group of higher education policies, which is historically what the department has focused on, on

student financial aid. The title for financial under title four,

it gives financial aid to institutions. So there is that line of issues and policy topics

related to higher ed at the same time. There was also a pretty robust

reform agenda at the K-12 level. So we’re talking about accountability, school choice, teacher quality.

All of those were at their height during the period that we’re studying at the tail

end. There’s been more of a focus on transition from K-12

to college and careers. Right. So that that kind of spanning the two historical

focused foci of the department has really started to emerge in the last few years talking

about college and career, ready standards, career and technical education.

And so I’d say those are the three main strands of debates that have been going on within the department

over the past decade or so. Are those debates simultaneous? Are they happening? Is the department

walking and chewing gum or is it. Is that surreal? Is it one thing than one thing than one thing?

Are largely simultaneous. What the. The higher ed policy debates happen almost completely

independently of the K-12 policy debates. And they’re both ongoing at the

same time. And I would also argue to put

a sort of a broader wrapper on this is that a lot of what we have initially

found is that a lot of the larger debates that you see occurring in the

public are not necessarily those most attended in the elite discussion of

education policy. So, for instance, I mean, are initial

findings indicate that these groups and citizens

by far the thing they care most about are governance issues. The debates going on

in the regulatory policy and setting up that framework aren’t really about

student scores or teacher quality. They’re about governance. They’re about the

structure of the governance of education. So who gets money? Who gets

to make decisions about how that money is spent? And so to build on that,

I think one of the bigger surprises of what we’ve seen so far is what is typically characterized as the policy

debates in the among the policy subsystem that the actors, the interest groups, the

advocacy organizations is not what’s playing out in comments on regulations.

We’re seeing some of the most influential groups being higher education

institutions and state education agencies in local school districts.

Right. Those aren’t considered advocacy organizations in any sense of the policy debate.

But they are highly influential in the regulatory comment process. So influential, in fact,

that whenever you sort of try to place groups or group

types in space, it’s these institutions that really structure the space for everyone else.

And that really gets to the heart of sort of the larger research

program and in understanding how organizations influence policy agendas

and the information supplied within them. I’d like to push you on that a little bit. I mean, I think if

you asked my name, who are the most important players in education policy debate? I mean, maybe I would name some of those

those education advocacy groups and those types of things. But I’ve asked an interest group scholar,

I feel like they would name the big institutions of higher education and the big, well-resourced, well-organized

groups. I mean, is it a surprise that they are they’re so prevalent and essentially people,

the stakeholders are so prevalent here or or or should

it be counterintuitive? So I think it would be surprising to beat up to people who participate in the policy

debates on a day to day basis. It might not be surprising to a political scientist, but if you go to D.C.

and talk to any of those think tank folks or any of the folks in the advocacy organizations, local school

districts don’t come to mind. Maybe teachers unions do. But state education agencies

there is far from their brain as as any group in structuring that

policy space. So why why do you think there’s the difference?

Well, I think there’s a difference, largely because the notice and comment, you know.

Period is not flashy. Right. If it’s not, what gets funders to grant you

money to do studies or to advocate for a given position. But it

is the process that really determines how how policy issues play out.

It also goes to a larger, I think, public misunderstanding of what regulations are.

So when we think when we said the word regulation, what most people think is a bureaucracy

somewhere is telling someone what to do, when to do it and how to do it. But the reality

is, when government goes, it gives resources, goods,

grants, assistance to anyone. It occurs and is almost

completely defined by regulatory policy. And in this process that we’re talking

about, even grants for things like FEMA play out over the course regulatory policy.

On this topic of these interest groups, I’m one of your findings that you sent us

showed all the different interest groups and the number of comments that they provided.

Is this is one an interest group provides more comments. Is that just

a result of the size of the interest group or are there some really small interest

groups with not a lot of resources, just putting out a lot of comments on education

policy? I think I think the way to think about it is not in terms of size.

And so if you sort of take some diversity scores, probably some you guys

would be familiar with, like the entropy or Hurford all indexes or something. What you find

is that the larger groups. It’s not so much that they’re able to comment

more frequently, but you have to understand the diversity so they can comment more

frequently across more topics. So the bigger organizations that

are better funded are more spread across topics, so they’re more fully engaged

in the policy process. The smaller groups tend to be have

more. We could say Nisha’s for commenting, right? So charter schools are

acutely focused on the issue of school choice. I think

that’s the that’s the. That’s the better way to sort of understand.

And some of these larger groups just I read them kind of off off your list here. Things like the National Education Association and

American Federation of Teachers. Right. No surprise there. Large teachers unions. You’ve got

the some version of a State Department of Education from states like Colorado,

Illinois, Ohio, New York, several others, and a couple of advocacy

groups in there. But for the most part, those those are the big ones. This

seems to be very focused on K-12 education to me, even though I want to show that a lot of the agenda is

focused, is his higher education. Am I right about that or. Yeah. And so

a lot of the agenda is higher education. The thing you have to understand, though,

is that it is education. All so back back when I wrote my book in 20 Tips

A. The largest producer of regulations

on education policy. This this was not in 2015.

But going back the previous decade was actually the Veterans Administration because they wrote all the

regulations for the G.I. Bill. So what we see and what our data tend to show

is this larger reform taking place, a shift in attention from

federal attention to higher education to K-12 education. I think Devin can

back me up. And I think it also reflects the politics during the Obama years, which was

if you think back to whatever the major education initiatives you hear, you think of Race to the Top. Right, which

was a K-12 initiative. You think of waivers from No Child Left Behind, you think

of teacher evaluation. And so all of that higher education dimension was still ongoing,

but there wasn’t any major reform. It was still just kind of chugging along as it had been

now. And in the more recent years, there’s been a greater focus on For-Profit Institutions. Right.

So our data collection might have not captured what we said. It was based on a sample

of of, you know, proof of concept for them for the grant, but it might not

have captured what was happening in the higher education dimension in more recent times.

I would also add to that that I think. One of the things you see over this period

provides us with an understanding of shifts, an issue attention that is these larger reform movements

have consequences for issue attention. You can think of them like plates on the earth when the

plates collide. You get things like subject seductions, you get mountains,

right. And so as these larger reform movements

collide. So so this fundamental division between higher ed or K-12

versus the reform movements focused on accountability, school choice and things

like this. What happens is some issues get pushed to the surface. Others

get pushed down below the surface. And so that all goes with this

larger understanding of the dynamics of the tension within the polls here. Do you think that student loan

at budget time series is a mountain out?

Leave that be so. Is that a partisan difference here? I think about the the

Obama Department of Education and yet Barack Obama appoints Arne Duncan, who’s

the Chicago school schools administrator, as his first education secretary. It’s very

focused on that. The types of issues the Democratic Party has obviously a long history with teachers,

with schools. I mean, this is a narrowly Democratic Party policy priority, K-12 education, but

it’s one where characterized by I mean stakeholders is one word, but by interest groups.

But then Betsy Devizes comes along, who is basically a Republican idealogue.

Right. She’s an American enterprises to fellow Kovalik. So my own research and.

What is the department going to take a very different the regulatory environment, going to take

a very different character under the conservative than under the liberal?

I think absolutely a will. It was actually, I think, quite remarkable how how much continuity

there was from W. George W. Bush to Barack Obama. That was remarkable

in how consistent the education policies were from 2001 to 2000,

Things have changed dramatically. I mean, we’ve already seen her scaling back the

the transgender guidance that the Obama administration provided. We’ve seen

new regulations on Title 9. We’ve seen new regulations

on any number of topics that are completely 180 degrees opposite

from what the Obama administration did. And so I think we will absolutely see a shift from the Obama to

Trump administration departments. So, for example, LBG LGBT groups might show up in the lobbying data

after 2017. I think that’s I think that’s probably right. I would also

add to that larger story and it builds on sort of this notion of continuity between

Bush and Obama is that it might you might be it might be a step too far to say we can we can

point out the secretary is the source of this variation. What we’re really talking about, there

is a set of presidents devoted to education reform, whether you agree with it or not,

versus one that is an avowed enemy of regulatory process.

And so there’s a substantive bit of this and there’s a procedural

bit of this in to build on what Sam said. I think the Obama administration’s reliance

on the regulatory process was unprecedented. They did a lot of policymaking

via guidance letters and the regulatory process ended in a manner that can easily

be undone by the Trump administration and secretary divorces department,

whereas Congress in this process. In the regulatory process.

Well, and specifically in the Department of Education over this time period. We have a very large education reform

passed in twenty, fifteen, fourteen, fifteen.

Well, I think that what Congress does is set the broad agendas

for bureaucracies. But if you think about the sort of stuff Congress

does, they’re giving broad form to the regulations

that you end up seeing in Department of Education that are then heavily influenced by

whoever the president is and whoever is her pick for the SEC. It

would be. But in terms of oversight, as with

all the other policy areas, there is none. And with

with Congress, they they did pass the Every Student Succeeds Act in 2015.

But I think a lot of what we’d see regulatory regulation wise is is quite similar, what

you saw to the after passage of No Child Left Behind, because the topics

involved in that legislation didn’t really change. It was just the substance around

them. The details about how often you had to test in what subgroups had to be put

together that were that were altered under ESSA. I think the other thing you have to remember is

that bureaucracies have to be responsive to problems on the ground as they emerge

and grants of authority to the bureaucracy or grants forever. The Department of Education can issue a regulation

pursuant to something from nineteen ninety five today, the same as something from 2015.

So thinking about going back pre No Child Left Behind

because I think we. I think you’re. What I’ve seen in education policy

makes a lot of sense for this being largely continuous since No

Child Left Behind. But I’m wondering before No Child Left Behind. Do you do? I know that

we don’t have the comments for back then. But do you just think,

based on what you know of education policy that, you know, post,

say, A, a nation at risk? Whenever the accountability movement

started, do you think we’re still going to have these interest groups acting in the same

ways and being involved in similar ways? Or do you think that there’s a difference

from. When these major laws passed.

So I think you can really delineate the education policy space at the federal

level with No Child Left Behind because that was the first time Title 1

funds were at risk if they didn’t follow the directives of regarding

standards, accountability and testing. Before that, states

were encouraged to do all those sorts of things. But most of them, many of them just simply ignored

it. And No Child Left Behind in the you know, using the stick of Title 1

funds is really what change the federal involvement in federal education

and on policy environment. And I would I would say before that you have a very long run

of history following the Second World War, where the federal government’s main focus had to

do with linking higher educational outcomes to economic

growth and things. So you had you had that sort of current and then I think the other

current was mostly folk focused on civil rights and civil

liberties concerns. Yeah, I think from 1965, from the initial Elementary and Secondary Education

Act through about a nation at risk at the K-12 level, it was almost purely equity considerations.

Nation at risk really started some focus on excellence that really came into being

with No Child Left Behind. And so I think, yeah, you might see a little of that show up between

a nation at risk and No Child Left Behind. But it was really No Child Left Behind that changed the

the the game regulatory. Why regulation wise at the federal level?

It was a it was a shift in focus that not only

not only presage sort of attention to the

secondary level of education, but also

jumbled the issues in the ways you can sort of observe in the data.

I suspect that, you know, if I if we could do the same analysis back in the 1990s

before No Child Left Behind, that would be a very different lobbying environment because those stakeholders probably come only after

the law passes where they chase the money rather than prompting Congress

to do so or they can’t. I guess I can’t from the Department of Education to do so. But those education

reformers that you expected to play a much larger role would be the ones really driving that change.

I think that’s right. I mean, I think, you know, prior to No Child Left Behind would be primarily state,

but departments of ad and school districts commenting on how funds should be distributed and allocated

and much less on substantive policy issues that they were going to be required to implement. And at the federal

level, a tremendous attention to things like the G.I. Bill and setting out

regulations that that sort of guide and how it would be dispensed to dispense it

and having banks, the major players as well. So as we as

we wrap up, I’d just like to ask you guys, what else what else should we read when we

when we think about education, education policy and regulatory agenda setting? What

what recent work besides your own work should our listeners find?

So Paul Mannah has done some tremendous work about the politics of education.

Collision Course is one book that really details the politics of No Child Left Behind nicely

prior to that. He has schools in which is about federalism in education.

Jeff Henig at Teachers College in Columbia, Columbia. He is he

does great work on the politics of that education as well, including the evolution

of the partisan nature of of education. And so those are

two main names that come to mind immediately. Susan Moffet at Brown is

another great scholar of education politics. And so those would be the ones that

I would offer. And also linking it to the broader regulatory process, something

I think, Susan, Yorkies work on interest group participation and commenting is good. And

in terms of how it structure rulemaking. Rachel Potter

at Virginia’s work is also very good. Now, I should actually add to what I usually don’t do here. I want my colleague

Merom Dwyer’s dissertation project on regular on lobbying and the regulatory process.

Coalitional lobbying is excellent and I think really, really fits into this literature. And Christina

Wallbrook and her student, I believe his name is Mike hearney wonderful,

wonderful project on the reframing of education in this period before the

No Child Left Behind Act that we were talking about, where you see just a real

shift in the frames that media use just to discuss education policy

and then a shift in the party platforms. But with that. Thank you guys very much for joining me. Katie,

thank you for joining me. This has been your policy agenda as podcast. Thanks for having us.

Thanks.