{"id":11,"date":"2018-03-12T16:52:33","date_gmt":"2018-03-12T16:52:33","guid":{"rendered":"http:\/\/podcasts.la.utexas.edu\/the-connector\/?post_type=podcast&#038;p=11"},"modified":"2020-11-16T19:01:39","modified_gmt":"2020-11-16T19:01:39","slug":"ep-04-polling-roundtable-pt-1","status":"publish","type":"podcast","link":"https:\/\/podcasts.la.utexas.edu\/the-connector\/podcast\/ep-04-polling-roundtable-pt-1\/","title":{"rendered":"Ep 04 &#8211; Polling Roundtable Pt 1"},"content":{"rendered":"<p>November 3, 1948. &#8220;Dewey Defeats Truman!&#8221; reads the bold front page headline of the Chicago Daily Tribune. But, of course, he had not. The polls had gotten it wrong. And if you keep up with the media, you might think polls continue getting it wrong. But do they? Are polls scientific? Are polls reliable?<\/p>\n","protected":false},"excerpt":{"rendered":"<p>November 3, 1948. &#8220;Dewey Defeats Truman!&#8221; reads the bold front page headline of the Chicago Daily Tribune. But, of course, he had not. The polls had gotten it wrong. And if you keep up with the media, you might think polls continue getting it wrong. But do they? Are polls scientific? Are polls reliable?<\/p>\n","protected":false},"author":13,"featured_media":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","episode_type":"audio","audio_file":"http:\/\/podcasts.la.utexas.edu\/the-connector\/wp-content\/uploads\/sites\/12\/2018\/03\/04-Polling-Roundtable-Pt-1.mp3","podmotor_file_id":"","podmotor_episode_id":"","cover_image":"","cover_image_id":"","duration":"","filesize":"35.78M","filesize_raw":"37523240","date_recorded":"","explicit":"","block":"","itunes_episode_number":"","itunes_title":"","itunes_season_number":"","itunes_episode_type":""},"tags":[13,16,11,14,9,10,12,15],"series":[2],"class_list":{"0":"post-11","1":"podcast","2":"type-podcast","3":"status-publish","5":"tag-campaigns","6":"tag-clinton","7":"tag-elections","8":"tag-politics","9":"tag-polling","10":"tag-polls","11":"tag-statistics","12":"tag-trump","13":"series-the-connector","14":"entry"},"acf":{"related_episodes":[{"ID":13,"post_author":"13","post_date":"2018-03-12 16:53:40","post_date_gmt":"2018-03-12 16:53:40","post_content":"It is the art of polling, not the science, which poses the biggest challenge. How should questions be worded? In what order should questions be asked? It turns out that the answers to questions such as these can create variation in polling results. We also discuss elections, with specific comments about the 2016 election cycle. Recorded in March, in the run-up to Super Tuesday, we explore what the polls might have been telling us about November 2016.","post_title":"Ep 05 - Polling Roundtable Pt. 2","post_excerpt":"","post_status":"publish","comment_status":"closed","ping_status":"closed","post_password":"","post_name":"ep-05-polling-roundtable-pt-2","to_ping":"","pinged":"","post_modified":"2020-11-16 19:00:46","post_modified_gmt":"2020-11-16 19:00:46","post_content_filtered":"","post_parent":0,"guid":"http:\/\/podcasts.la.utexas.edu\/the-connector\/?post_type=podcast&#038;p=13","menu_order":0,"post_type":"podcast","post_mime_type":"","comment_count":"0","filter":"raw"}],"hosts":[{"ID":70,"post_author":"49","post_date":"2020-07-16 19:26:16","post_date_gmt":"2020-07-16 19:26:16","post_content":"<!-- wp:paragraph -->\n<p>TBD<\/p>\n<!-- \/wp:paragraph -->","post_title":"Stuart Tendler","post_excerpt":"","post_status":"publish","comment_status":"closed","ping_status":"closed","post_password":"","post_name":"stuart-tendler","to_ping":"","pinged":"","post_modified":"2020-07-16 19:26:17","post_modified_gmt":"2020-07-16 19:26:17","post_content_filtered":"","post_parent":0,"guid":"http:\/\/podcasts.la.utexas.edu\/the-connector\/?post_type=speaker&#038;p=70","menu_order":0,"post_type":"speaker","post_mime_type":"","comment_count":"0","filter":"raw"}],"guests":[{"ID":76,"post_author":"49","post_date":"2020-07-16 19:35:27","post_date_gmt":"2020-07-16 19:35:27","post_content":"<!-- wp:image {\"align\":\"center\"} -->\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img src=\"https:\/\/minio.la.utexas.edu\/colaweb-prod\/person_files\/0\/287\/shaw_200.jpg\" alt=\"Daron Shaw\" \/><\/figure><\/div>\n<!-- \/wp:image -->\n\n<!-- wp:paragraph -->\n<p>Professor Shaw earned his Ph.D. from the University of California, Los Angeles. Prior to accepting a position at the University of Texas in the fall of 1994, he worked in several political campaigns as a survey research analyst. Professor Shaw also served as a strategist in the 2000 and 2004 presidential election campaigns. His research and teaching interests include American Government, Campaigns and Elections, Political Parties, Public Opinion and Voting Behavior, and Applied Survey Research. He is co-director of the Fox News Poll, co-director of the University of Texas\/Texas Tribune Poll, director of the Texas Lyceum Poll, and associate Principle Investigator for the 2020 American National Election Study. Professor Shaw is also a member of the national decision team for Fox News, the advisory board for the MIT Election Data &amp; Science Lab, the advisory board for the Annette Strauss Institute, and the editorial board for Political Behavior. Formerly, he served as President George W. Bush\u2019s representative on the National Historical Publications and Records Commission and as one of the academic directors for President Barack Obama\u2019s Commission for Election Administration.<\/p>\n<!-- \/wp:paragraph -->\n\n<!-- wp:paragraph -->\n<p>Professor Shaw\u2019s most recent book is&nbsp;<em>The Turnout Myth<\/em>&nbsp;(co-authored with John Petrocik), published by Oxford University Press. His first two books,&nbsp;<a href=\"http:\/\/www.oup.com\/us\/catalog\/general\/subject\/Politics\/AmericanPolitics\/ElectionsPublicOpinionVotingBeha\/~~\/dmlldz11c2EmY2k9OTc4MDE5NTM2NjgzOQ==\"><em>Unconventional Wisdom: Facts and Myths about American Voters<\/em><\/a>&nbsp;(Oxford Press, co-authored with Karen Kaufmann and John Petrocik) and&nbsp;<a href=\"http:\/\/www.press.uchicago.edu\/presssite\/metadata.epl?mode=synopsis&amp;bookkey=203888\"><em>The Race to 270<\/em><\/a>&nbsp;<em>(Chicago Press),<\/em>&nbsp;were published in 2008 and 2006, respectively. In addition, Professor Shaw has published articles in the leading journals in the discipline, including&nbsp;<em>American Political Science Review, American Journal of Political Science, Journal of Politics, British Journal of Political Science, Political Research Quarterly, Political Behavior, Political Communication, PS: Political Science, Party Politics, Presidential Studies Quarterly<\/em>, and&nbsp;<em>American Politics Research<\/em>.<\/p>\n<!-- \/wp:paragraph -->","post_title":"Darren Shaw","post_excerpt":"","post_status":"publish","comment_status":"closed","ping_status":"closed","post_password":"","post_name":"darren-shaw","to_ping":"","pinged":"","post_modified":"2020-07-16 19:35:29","post_modified_gmt":"2020-07-16 19:35:29","post_content_filtered":"","post_parent":0,"guid":"http:\/\/podcasts.la.utexas.edu\/the-connector\/?post_type=speaker&#038;p=76","menu_order":0,"post_type":"speaker","post_mime_type":"","comment_count":"0","filter":"raw"},{"ID":73,"post_author":"49","post_date":"2020-07-16 19:30:58","post_date_gmt":"2020-07-16 19:30:58","post_content":"<!-- wp:image {\"align\":\"center\"} -->\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img src=\"https:\/\/minio.la.utexas.edu\/colaweb-prod\/person_files\/0\/843\/robert_luskin_profile_image.jpeg\" alt=\"Robert Luskin\" \/><\/figure><\/div>\n<!-- \/wp:image -->\n\n<!-- wp:paragraph -->\n<p>Professor Luskin has also taught at the University of Alabama, Indiana University,&nbsp;<em>l'Universit\u00e9 de Paris I<\/em>&nbsp;(<em>la Sorbonne<\/em>), Princeton University,&nbsp;and Stanford University and in the ICPSR Summer Program at the University of Michigan, the ECPR Summer School at the University of Essex, and the Summer School on Advanced Methods in the Social Sciences at the&nbsp;<em>Universit\u00e0 della Svizzera Italiana<\/em>.&nbsp; He has been a Fellow at the Center for Advanced Study in the Behavioral Sciences at Stanford and&nbsp;<em>Chercheur Associ\u00e9<\/em>&nbsp;at the&nbsp;<em>Centre d'Etude de la Vie Politique Fran\u00e7aise<\/em>&nbsp;in Paris.&nbsp; He is&nbsp;a Research Advisor at the Center for Deliberative Democracy at Stanford University and Director of the Center for Deliberative Opinion Research at the University of Texas at Austin. He has been a member of the Advisory Board of the Texas Poll and of the Editorial Boards of&nbsp;<em>Political Analysis<\/em>&nbsp;and the&nbsp;<em>American Political Science Review<\/em>.<\/p>\n<!-- \/wp:paragraph -->\n\n<!-- wp:paragraph -->\n<p>His general interests include public opinion, voting behavior, political psychology, and statistical methods, and he has long been particularly interested in the effects of political information on the texture and outcomes of representative democracy.&nbsp; Among other projects, he is using Deliberative Polling in the U.S. and abroad to examine the empirical dimensions of deliberative democracy and is working on a study of political information in France.&nbsp; He has published papers on these and other topics in the&nbsp;<em>American Political Science Review<\/em>, the&nbsp;<em>American Journal of Political Science<\/em>, the&nbsp;<em>Journal of Politics<\/em>, the&nbsp;<em>British Journal of Political Science<\/em>,&nbsp;<em>Political Analysis<\/em>, and other scholarly journals.<\/p>\n<!-- \/wp:paragraph -->","post_title":"Robert Luskin","post_excerpt":"","post_status":"publish","comment_status":"closed","ping_status":"closed","post_password":"","post_name":"robert-luskin","to_ping":"","pinged":"","post_modified":"2020-07-16 19:30:59","post_modified_gmt":"2020-07-16 19:30:59","post_content_filtered":"","post_parent":0,"guid":"http:\/\/podcasts.la.utexas.edu\/the-connector\/?post_type=speaker&#038;p=73","menu_order":0,"post_type":"speaker","post_mime_type":"","comment_count":"0","filter":"raw"},{"ID":79,"post_author":"49","post_date":"2020-07-16 19:42:24","post_date_gmt":"2020-07-16 19:42:24","post_content":"<!-- wp:image {\"align\":\"center\"} -->\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img src=\"https:\/\/minio.la.utexas.edu\/colaweb-prod\/person_files\/0\/5341\/christopher_wlezien_profile_image.jpeg\" alt=\"Christopher Wlezien\" \/><\/figure><\/div>\n<!-- \/wp:image -->\n\n<!-- wp:paragraph -->\n<p>Christopher Wlezien is Hogg Professor of Government. He joined the University of Texas faculty in 2013 from Temple University in Philadelphia. Previously he taught at Oxford University, where he was Reader of Comparative Government and a Fellow of Nuffield College. While at Oxford, he co-founded the ESRC-funded Oxford Spring School in Quantitative Methods for Social Research. Before that, he taught at the University of Houston, where he was founding director of the Institute for the Study of Political Economy. He holds or has held visiting positions at Academia Sinica (Taiwan), Australian National University, Columbia University, University of Copenhagen, European University Institute (Florence), Instituto Empresa (Madrid), Juan March Institute (Madrid), University of Mannheim (Germany), McGill University (Montreal), Sciences Po (Paris), and the University of Manchester (UK). He received his Ph.D. from the University of Iowa in 1989 and his B.A. from Saint Xavier College (Chicago) in 1984.<br><br>His primary, ongoing research develops a \u201cthermostatic\u201d model of public opinion and policy and examines the dynamic interrelationships between preferences for spending and budgetary policy in various domains. A cross-national investigation focusing on the US, the UK, and Canada is the subject of a book titled&nbsp;<em><a href=\"http:\/\/www.cambridge.org\/catalogue\/catalogue.asp?isbn=9780521868334\">Degrees of Democracy<\/a><\/em>, published by Cambridge University Press. A more recent&nbsp;<a href=\"https:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/01402382.2012.713752\">article<\/a>&nbsp;tests theories about the effects of federalism, executive-legislative imbalance, and the proportionality of electoral systems in 17 countries. A related volume on&nbsp;<em><a href=\"http:\/\/www.russellsage.org\/publications\/who-gets-represented\">Who Gets Represented?<\/a><\/em>, published by the Russell Sage Foundation, investigates representational inequality in the US and stimulated a more recent&nbsp;<a href=\"https:\/\/onlinelibrary.wiley.com\/doi\/full\/10.1002\/polq.12577\">article<\/a>. Current research in the area considers&nbsp;<a href=\"https:\/\/onlinelibrary.wiley.com\/doi\/abs\/10.1111\/psj.12285\">news coverage<\/a>&nbsp;and how it&nbsp;<a href=\"https:\/\/journals.sagepub.com\/doi\/abs\/10.1177\/1940161219832416\">mediates<\/a>&nbsp;public responsiveness to policy, and there is related&nbsp;<a href=\"https:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/10584609.2020.1763529?journalCode=upcp20&amp;\" target=\"_blank\" rel=\"noreferrer noopener\">work<\/a>&nbsp;comparing dictionary and supervised learning methods to automated content analysis. A book on the broader subject, entitled&nbsp;<em>Information and Democracy: Public Policy in the News<\/em>, is under contract with Cambridge University Press.<\/p>\n<!-- \/wp:paragraph -->\n\n<!-- wp:paragraph -->\n<p>His other major area of research addresses the evolution of voter preferences expressed in pre-election polls over the course of an election cycle. It has been the subject of numerous articles on the US and a book&nbsp;<em><a href=\"http:\/\/press.uchicago.edu\/ucp\/books\/book\/chicago\/T\/bo13948250.html\">The Timeline of Presidential Elections<\/a><\/em>&nbsp;that was published in 2012 by the University of Chicago Press. A related e-book&nbsp;<em><a href=\"http:\/\/press.uchicago.edu\/ucp\/books\/book\/chicago\/Other\/bo19211950.html\">The 2012 Election and the Timeline of Presidential Elections<\/a><\/em>&nbsp;was published in 2014 and there is similar&nbsp;<a href=\"https:\/\/journals.sagepub.com\/doi\/abs\/10.1111\/1467-9248.12008\">research<\/a>&nbsp;on the UK.&nbsp; His current work in the area undertakes cross-national analysis, the first&nbsp;<a href=\"https:\/\/onlinelibrary.wiley.com\/doi\/abs\/10.1111\/ajps.12189\">article<\/a>&nbsp;on which examines how political institutions condition the structure and evolution of electoral preferences in more than 300 elections in over 40 countries. A related&nbsp;<a href=\"https:\/\/www.nature.com\/articles\/s41562-018-0315-6\">article<\/a>&nbsp;assesses pre-election poll errors in those elections, data for which is publicly available at my poll datasets&nbsp;<a href=\"https:\/\/liberalarts.utexas.edu\/government\/faculty\/cw26629#datasets-on-polls-and-the-timeline-of-elections\">site<\/a>. Recent methodological&nbsp;<a href=\"https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0261379416302980\">research<\/a>&nbsp;explores issues in the application of the \u201ctimeline\u201d approach to studying electoral dynamics, and points to an approach for simultaneously assessing the effects of different system, party, and election-level variables.&nbsp;&nbsp;&nbsp;<\/p>\n<!-- \/wp:paragraph -->\n\n<!-- wp:paragraph -->\n<p>He has produced other substantive and methodological research on a variety of related topics, including electoral institutions and representation, mass media and public responsiveness to policy, public perceptions of political parties, economic voting, public opinion polling, election prediction and forecasting, the mass media and election campaigns, mass media coverage of the economy, policy voting, and time series analysis, all of which is listed on his&nbsp;<a href=\"https:\/\/minio.la.utexas.edu\/colaweb-prod\/person_files\/0\/5341\/Wlezien%20CV,%20April,%202020,%20for%20posting.pdf\">CV<\/a>&nbsp;and posted&nbsp;<a href=\"https:\/\/liberalarts.utexas.edu\/government\/faculty\/cw26629#publications\">here<\/a>.<\/p>\n<!-- \/wp:paragraph -->\n\n<!-- wp:paragraph -->\n<p>Wlezien was founding co-editor of the international&nbsp;<em>Journal of Elections, Public Opinion and Parties<\/em>&nbsp;and currently is Associate Editor of&nbsp;<em>Public Opinion Quarterly, Research and Politics,<\/em>&nbsp;and&nbsp;<em>Parliamentary Affairs<\/em>&nbsp;and also a member of the editorial boards of six other journals. He is President-Elect of the&nbsp;<a href=\"http:\/\/spsa.net\/\">Southern Political Science Association<\/a>&nbsp;after chairing the program for the 2019 annual meeting held in Austin. &nbsp;At the University of Texas, he is a faculty affiliate of the&nbsp;<a href=\"http:\/\/www.policyagendas.org\/\">Policy Agendas Project<\/a>&nbsp;and the&nbsp;<a href=\"https:\/\/www.utexas.edu\/cola\/centers\/european_studies\/\">Center for European Studies<\/a>.&nbsp;&nbsp;<\/p>\n<!-- \/wp:paragraph -->","post_title":"Christopher Wlezien","post_excerpt":"","post_status":"publish","comment_status":"closed","ping_status":"closed","post_password":"","post_name":"christopher-wlezien","to_ping":"","pinged":"","post_modified":"2020-07-16 19:42:24","post_modified_gmt":"2020-07-16 19:42:24","post_content_filtered":"","post_parent":0,"guid":"http:\/\/podcasts.la.utexas.edu\/the-connector\/?post_type=speaker&#038;p=79","menu_order":0,"post_type":"speaker","post_mime_type":"","comment_count":"0","filter":"raw"}],"transcript":"<p>[0:00:01 Speaker 0] November 3rd. 1948 Dewey Defeats Truman, reads the bold front page headline of the Chicago Daily Tribune. But of course he had not. Polls have gotten it wrong. And if you keep up with the media, you might think polls continue getting it wrong. But do they are pole scientific? Our polls reliable. Welcome back to the connector, where we bring together innovative, groundbreaking and collaborative research inside the U. T. Austin political science universe. I&#8217;m your host, Stewart Tendler. In this episode, we talked with three political scientists. You know quite a lot about polling. We asked Robert Luskin, Darren Shaw and Chris Malaysian whether complaints about polling methods are valid. Bucking popular perception, our guests suggests that such complaints are misleading, often missing the point or ignoring inconvenient truths. In fact, in important ways, polls are becoming more accurate. But why did the media pay attention of polls? Regardless, is reporting on polls the equivalent of a news media junk food binge or a serious component feeding their storytelling needs? We close this episode with an in depth statement about the margin of error in polling and an explanation as to why the media often misreads this statistic. Why don&#8217;t we just start with everybody briefly introducing themselves, letting us know who who&#8217;s here?<\/p>\n<p>[0:01:31 Speaker 3] Bob Luskin. I&#8217;m a professor here at U T and, uh, have co written actually a on article about pre election media polls with Darren and Mark Heatherington, Vanderbilt. I&#8217;m also advising the French national election Study.<\/p>\n<p>[0:01:48 Speaker 2] I&#8217;m Daryn Shaw. I&#8217;m a professor here at University of Texas at Austin Um, co director of the Fox News Poll, along with Chris Anderson of Anderson Robbins Research. Um, one of the co p eyes principle investigators for the Texas Tribune, University of Texas Government Department. Pole is a regular statewide poll of Texas, and I&#8217;m a member of the Fox News Decision team.<\/p>\n<p>[0:02:10 Speaker 1] I&#8217;m Chris Pleasant, also professor of government here at University of Texas at Austin. And I do work out polling of all sorts, really, from vote intentions over the course of campaigns to public preferences for policy and the relationship with what policymakers actually do.<\/p>\n<p>[0:02:28 Speaker 0] Great. Well, thanks to each of you for being here, I figured it&#8217;s a presidential election year. It would make sense to talk to three people who know a lot more than a lot of people about polls. I wanted to start off. So I have this article from by Jill LePore. That was in the November 2015 issue of The New Yorker. And the little subtitle says Polling may never have been less reliable form or influential than it is now. And then if I get fast forward to the very end of the article quoting Donald Trump, she writes, Do we love these polls? He called out to a crowd in Iowa. Somebody said You love polls, I said. That&#8217;s only because I&#8217;ve been winning every single one of them, right? Right. Every single poll Two days later, when he lost his lead in Iowa to Ben Carson, he&#8217;d grown doubtful. I honestly think those polls were wrong. By the week of the third GOP debate, he&#8217;d fallen behind in a national CBS New York Times poll. The thing with these polls are also different, Trump said mournfully. It&#8217;s not very scientific, so that would be the first question I would pose our polls Scientific are poles reliable? Well,<\/p>\n<p>[0:03:39 Speaker 2] I mean, that&#8217;s the question. Is it? I I&#8217;m amused by this story just because my experience in polling has been that when someone doesn&#8217;t like the result that occurs in a particular poll, they immediately become experts on methodology and in fact, difference to Bob and Chris. I&#8217;m thinking that my middle name ought to be changed to dear and flawed methodology Shaw. So I I always take these complaints about methodology with the large grain of salt. Now, having said that, the tough thing when I hear those sorts of complaints and arguments and observations on Poles is it. There are a lot of issues in contemporary polling, important issues about both the methodology about the underlying populations that were trying to survey into to get insight into, Ah, but but the ones that tend to be identified by candidates or advocates of particular policy positions tend not to be the ones that I think really are of interest to those of us who do this stuff for a living, you know, So and we could elaborate on this quite a bit, I&#8217;m sure. But you know questions about the advent of on line polling, about the need for cellphone supplements to traditional polling about likely voter samples about list based samples. These are things that are really important issues in the industry and, you know, do lead to issues that can bias poles and produced results that don&#8217;t square very well with election results. But you know these things, they&#8217;re kind of treated in a not very detailed manner. For obvious reasons. I think when it comes to you know, the public presentation of these things,<\/p>\n<p>[0:05:09 Speaker 3] the context of that article I take it waas the nomination campaign. And it is true that polls of candidate preferences in nomination campaigns are less reliable than corresponding polls in the general election, where party identification ties people down makes makes their choices much more predictable. So you don&#8217;t have that, of course, to distinguish between the candidates and in a nomination campaign. That, and that means that people&#8217;s preferences are much more lay bile. And it&#8217;s, uh, there&#8217;s a greater element of unpredictability.<\/p>\n<p>[0:05:48 Speaker 1] Yeah, one of the things reflecting on both what their and Bob have said. The polling used to be really, really simple. Everybody used one single methodology and one pretty identical, very similar question. They would get their their samples in a pretty similar way, and they would administer it in essentially the same way face to face we all went to phone, and we&#8217;ve gone now from ah, maybe having one standard or perhaps to toe having many, many different standards. And there&#8217;s not to say that these air, not scientific polls. Well, maybe there are different sciences and these air maybe being practice sciences, air being practised to a varying degree. Which is all to say. I think when we&#8217;re in the lesson, I think I tried emphasize with people and I think Darren was reflecting on this and Bob has in the past with me as well that it&#8217;s really makes it less harder and harder for us to rely on the results of any single pole or poles of particular survey houses. Which leads us to rely on, you know, aggregations we sometimes refer to as the poll of polls On that way. We&#8217;re sort of, you know, we&#8217;re not beholden to the decisions of a particular organization or a particular type point. There is an assumption there that these kind of errors that air in this polling, which is always the case, there&#8217;s always Aaron polling kind of cancel out. But which may not be the case that maybe there&#8217;s a systematic bias But<\/p>\n<p>[0:07:06 Speaker 2] there&#8217;s some. There&#8217;s some really interesting examples like that get played up a lot. Sometimes they&#8217;re right. Sometimes they&#8217;re not. So you&#8217;ll hear a lot of people comment on the the Scottish independence vote. Last elections in Great Britain, the Israeli elections as well, some high profile elections in the United States, the Kentucky race from last cycle the Virginia raised from 2014 where the the conventional wisdom is the polls were massively off. One I think that&#8217;s overstated. They actually weren&#8217;t is far off. Is a lot of people popular. They think the Virginia case may be an exception to that, the one where the Republican was significantly understated but actually weren&#8217;t that far off in Britain and in the case, This guy&#8217;s referenda in Israel. And in fact, and there&#8217;s also the question of aggregating, you know, a question that basically asked what we called united some generic ballot test, you know, who were gonna vote for for Congress of the Legislature, translating that into seats Aziz, you do in Great Britain or in Israel. And so there is some of these questions of aggregation, but more fundamentally they weren&#8217;t that far off. And yet There&#8217;s been this tale told about how inaccurate the polls are. And some of the work that that Bob and I have done with Mark Heatherington has suggested a couple things. One even today, and we&#8217;re still collecting data from last couple of cycles, but they&#8217;re not that far off. I think the latest estimate we have Bob is a point of half or something like that, and they&#8217;re not off. Maybe even more importantly, for consumers out there, they&#8217;re not consistently off with respect to direction. There&#8217;s a slight tendency over time. And Bob, you may remember that LeBaron idea.<\/p>\n<p>[0:08:35 Speaker 3] There&#8217;s more than one branch to this research. The one we&#8217;ve been focusing on has to do with the signed error that is taking account. We&#8217;re looking at not just how far the pole is off, but which way it airs toward the Democratic or the Republican. But another branch, which were not actively working out at the moment, but we&#8217;ve looked at, has to do with the absolute magnitude of the air and at least in our data, which do not include the last few cycles were working on that, at least in our data, there&#8217;s been a very noticeable downward trend in the error and even the absolute terror in pre election polls. They&#8217;ve been getting at least up to the point of our data stopped. For the moment, they have been getting more accurate despite the handicaps that pollsters face. I wanted to pick up on something that Chris said, and one on Darren just addressed it a swell that this has to do with the treatment of polls in the media. I think one reason for this overstatement the boat of the degree of error that both Darren and Chris pointed to is that the media do not generally focus on the average of polls, which is, I agree with Chris is what they should be doing. Instead, they take the polls one by one, and the latest poll, whatever it is, is gospel. Uh, you know, that&#8217;s the way things are. Even if it&#8217;s an outlier. That&#8217;s the way things are. And of course, it&#8217;s the most vivid examples of, especially in retrospect, looking at the election results. It&#8217;s the most vivid examples that remain in the media&#8217;s memory, so they, you know, they remember the cases that were far off for getting the ones that were close or off in the other direction. So the averages is still pretty close. But what they remember is the ones that were way off in general. I could not agree more than what the media should be doing is not focusing on each pole as it comes out, but reporting, Ah, running average. You know, the polls done inappropriate period, they will be more accurate. And, ah than any single poll if you take the average.<\/p>\n<p>[0:10:34 Speaker 2] Yeah, I think that&#8217;s right. In fact, I think the aggregate statistics from 2012 and 2008. Ah, and I&#8217;m referring here toe prints of some of the work that Nate Silver is done on fivethirtyeight dot com, where he predicts, basically, using these averages. You know, there&#8217;s some question about how he waits them. But even if you just done, say, the real clear politics average of polls that I believe you would have gotten 49 under 50 states in one election and 50 out of 50 and the other and thats with no complicated waiting or anything like that, So so I agree with Chris and Bob about how you know there are of course airs in these polls, but not that far off. And certainly collectively they really give you pretty instructive picture about what&#8217;s going on.<\/p>\n<p>[0:11:13 Speaker 1] But most of I think what we&#8217;re talking about here pertains to general elections<\/p>\n<p>[0:11:17 Speaker 2] is true. That&#8217;s right. That&#8217;s<\/p>\n<p>[0:11:19 Speaker 1] true. Does it? Certainly.<\/p>\n<p>[0:11:20 Speaker 3] Our research is strictly general election, right?<\/p>\n<p>[0:11:22 Speaker 1] Right and minus is mostly focused on that as well. It&#8217;s a lot. It&#8217;s a lot trickier when we&#8217;re facing Ah, trying to predict primary. Yeah,<\/p>\n<p>[0:11:30 Speaker 2] And then there&#8217;s the question of using national poll results versus state by states, and some of these states have become very important. Every four years in the presidential cycle tend to be under polled. You know, I don&#8217;t know what a good sample of Vermont looks like because it&#8217;s never competitive at the senatorial level, and it&#8217;s never competitive the presidential level. But a couple weeks ago, we were all very interested in the Vermont primary and whether Kasich had closed on Trump and turns out he had. Although I don&#8217;t know that any polls necessarily got that right, It&#8217;s, you know, I&#8217;m I&#8217;m sort of fairly impressed by polls of Ohio or Florida because there&#8217;s so many you know institutions and organizations that do it, and you can kind of judge whether one&#8217;s an outlier. But if, say, North Dakota ever becomes a battleground state again, I have very little confidence that were we have the expertise or background, or they will be the number of poles necessary to get it right.<\/p>\n<p>[0:12:19 Speaker 1] I think they did identify the Republican caucus scholars.<\/p>\n<p>[0:12:23 Speaker 3] That&#8217;s our polls anticipating caucus votes or even more apt to be off than ones devoted primaries.<\/p>\n<p>[0:12:31 Speaker 2] Yeah. I mean, look, if you identify 400 Republicans from some lists registered voter list and then you figure okay, 15% of them are gonna vote. You know how you get to that? 15% is one issue, and then another is, Well, if I&#8217;ve got 15% of 400 I don&#8217;t have a very large sample from which get a very accurate yeah, very accurate picture of who&#8217;s of what the distribution of the vote&#8217;s gonna be.<\/p>\n<p>[0:12:56 Speaker 3] You know, there&#8217;s a general point that, ah least in my view, ought to be aired since we&#8217;re talking about polls and some degree about the media&#8217;s use of them. And that is to what extent normative lee, the media should be so obsessed with polling, it&#8217;s ah or indeed should be reporting polls at all. In my view, this is the junk food of political discourse. It&#8217;s ah, free of nutritive value. It&#8217;s, ah, what people should be talking about and what what the media should be letting the candidates conveyed to us is their policy positions. That&#8217;s that&#8217;s what people should be voting on to focus so much on polling. I understand why they do it. You know, it&#8217;s it&#8217;s a way of avoiding any risk of of bias. You know, this is science. We can point to the numbers in polling and makes reporting easier, but it&#8217;s not useful to voters and made maybe affirmatively unuseful to voters. It may need people to vote for the candidate who looks as though he or she is going to win, and that&#8217;s not the grounds on which people in a democracy should be voted<\/p>\n<p>[0:13:59 Speaker 1] well. But we do know there&#8217;s I completely agree. We do know there&#8217;s work showing that people love these polls, right. Readers of the newspaper opportunity to choose between the latest right horse race coverage and the next installment on some policy issue, they tend to gravitate to the former.<\/p>\n<p>[0:14:16 Speaker 2] But one thing you do realize when you when you see the other side of the picture that is from the media&#8217;s point of view and II under appreciated this ah, earlier in my career. But I have a pretty decent appreciation for it now. Is there need to tell stories? So you conduct a poll and you you give the top line results. You say Trump is winning by 10 points. And then you start telling stories about angry voters, about disaffected voters, about uneducated voters. And, you know, the more content there is out there with the Web and with 24 hour cable, the more they need there is to tell stories. I mean, when I&#8217;ve started doing decision teamwork and exit polling work, I thought the top of the hour call was the big deal. You know that we would say whether it can, I want to stay or not. And it turned out no. The top of the hour call last about one minute, and then they got another 59 minutes, you know, minus told minutes for commercials to talk about something. And so it&#8217;s what the exit polls do certainly on election night, and I think probably these other polls to is they allow you to, you know, not just say it trumps winning, but give me some sort of story and is we&#8217;ve suggested sometimes that story&#8217;s not terribly correct in some sense, but sometimes it is. But they need to tell a story.<\/p>\n<p>[0:15:25 Speaker 3] Well, this story also influences the way they present the polling numbers. I remember noticing, I don&#8217;t remember exactly which candidates were involved. But earlier in this present nomination campaign, there was a poll on both Republicans and Democrats, and reporting on was no Candidate X is ahead of candidate why on the Republican side, by nine points exclamation point. And then there was a nine point gap between candidates Z and W. On the Democratic side, on the way that was phrased was, well, candidate ZIS is only ahead of candidate W by nine points. You know, it was the same numerical difference, but that&#8217;s what fit their narrative,<\/p>\n<p>[0:16:14 Speaker 1] coming back to the point you made about the trial heat focused on the horse race and all that, um, and how that might not be the most important thing for them to be focusing on. I don&#8217;t mean the chauffeur organization, but there&#8217;s a big, well known survey organization by the name of Gallup, which would agrees with you. In fact, they&#8217;ve abandoned, um that part of their polling and actually increased their polling of substandard opinion. A striking change<\/p>\n<p>[0:16:38 Speaker 3] there. Also, several countries that have unsuccessfully band either pre election polls or the publication of them, especially because of this.<\/p>\n<p>[0:16:46 Speaker 2] I&#8217;m all in favor of a black market for Poles. I&#8217;d like to fill that niche. I have this piracy kind of thing about it. I think that&#8217;s kind of cool, actually. Renegade pollsters. But But I don&#8217;t want to go to prison for polling. The one thing I&#8217;ll say and I don&#8217;t disagree with what Bob said about, I think was a junk food quality of the ballots. There&#8217;s a literature in with respect to nomination politics or seek presidential primaries that talks about how people use information about viability and viability. Meaning, you know, chances that a candidate is is got a chance to actually win the nomination, and this is what the early contests do. This is what polling has kind of done. It allows Republicans for instance to discriminate between Carly Fiorina and Jeb Bush and Ted Cruz and Marco Rubio and others. Now, I I don&#8217;t know that I think that using that information is necessarily a good thing. But I&#8217;m not sure it&#8217;s not a good thing for those voters. You know, is Is it useful for me to, you know, look up in a Texas Poland, find that candidate I might have supported has zero support, but someone who is reasonably close to my preferences has ah chance to win. This is a unique condition the United States of presidential politics. But I&#8217;m trying to think of exceptions, maybe to the junk food rule, and<\/p>\n<p>[0:18:04 Speaker 1] apply some multiparty system doesn t.<\/p>\n<p>[0:18:08 Speaker 3] So yeah, so I do think that&#8217;s a benefit of polling information, assuming a reasonable degree of accuracy. But it doesn&#8217;t negate or, in my view, fully counterbalanced the the problematic side of things. So, sure, it makes it easier if you&#8217;d to judge which of two candidates you think you largely agree with, you should vote for on the basis of which has the greater chance of winning the election of getting the nomination. But it makes it much harder for you to judge who you generally agree with. If they&#8217;re not getting policy information, can I change the subject? Still staying with polling, of course. But the I didn&#8217;t want our time to elapse without my getting to mention, uh, probably my greatest dissatisfaction with reporting of polls, which is the use of the margin of error to dismiss many results as a quote statistical tie when there no such thing. There&#8217;s a kind of twisty logic behind this from classical statistical inference that if you assume that two candidates are actually tied and there&#8217;s a margin of error of, say, 4% that means that the probability of seeing and observed difference between them of greater than 4% is less than 40.5 on one in 20 chance. If the difference between them is less than 4% then there is a greater than 40.5 chance of that&#8217;s occurring by chance, right? The question is, does this Since any sample is a random draw, ideally a random draw from the population, the results will always be different from what&#8217;s true of the population in some degree. Occasionally it will be very different just by chance So the question is, what&#8217;s the probability of seeing what you see if their candidates are really tied? But if the margin of error is 4% and the candidates or two or 3% apart, it is still very unlikely, not 0.5 that maybe 0.2 or something like that. Still very unlikely that you&#8217;d see those results by chance. The candidates who is, say, two points ahead with a four point margin of Eric is very likely ahead, and to dismiss it as a statistical tie really is wrong.<\/p>\n<p>[0:20:19 Speaker 1] It&#8217;s another argument for pulling the polls as well. If you&#8217;ve noticed the tendency, what one poll comes out, 500 respondents or so there&#8217;s a 3% difference. But it&#8217;s not significantly statistically significant. We have another one, which was a 4% difference. Not significant. Another one with three. If we add them together, we easily find them.<\/p>\n<p>[0:20:38 Speaker 2] This is Yeah, this is my Ohio example from 2012 where President Obama I was up in, I want to say 10 consecutive balls, all within margin of error on dso you. You had people saying like Well, this is within March of Aaron, but if you calculate the chances that Obama is behind When you when you have seen 10 consecutive surveys showing him ahead by one point, even it&#8217;s it&#8217;s infinite testable. So you know, in fact, I think about that logically, right? If if if we are actually tied or if you know Romney is ahead, what are the chances that I&#8217;m gonna draw 10 straight polls showing the opposite? That the head? Yeah, yeah, exactly.<\/p>\n<p>[0:21:14 Speaker 3] And that is a common scenario. Frequently, in a close election, you will find a series of polls, all with results within the margin of Arab, but all pointing in the same direction. And that&#8217;s because each of them is probably individually. The results showing Candidate X with slim lead should be taken seriously. And, uh, and evidence of that is that Candidate X is a slim leading. You know, five other polls<\/p>\n<p>[0:21:39 Speaker 1] and it is try as we might, there&#8217;s there&#8217;s a riel incentive, not toe reflect that in a lot of reporters write. It makes the race a lot less interesting to say that Obama has Ohio in the<\/p>\n<p>[0:21:49 Speaker 0] PAC s. Oh, are you convinced about the margin of error? Thanks for listening to this episode of the connector where we dove into the world of polling. If you are anything like me, you&#8217;ll be thinking about polls just a little bit differently from now on. Stay tuned for Part two of this episode when we turn our attention toward November 2016. If you have any questions or comments, you can find me a Gov dot utexas dot edu<\/p>\n"},"episode_featured_image":false,"episode_player_image":"https:\/\/podcasts.la.utexas.edu\/the-connector\/wp-content\/uploads\/sites\/12\/2018\/03\/mza_586582831322042141.jpg","download_link":"https:\/\/podcasts.la.utexas.edu\/the-connector\/podcast-download\/11\/ep-04-polling-roundtable-pt-1.mp3","player_link":"https:\/\/podcasts.la.utexas.edu\/the-connector\/podcast-player\/11\/ep-04-polling-roundtable-pt-1.mp3","audio_player":"<audio class=\"wp-audio-shortcode\" id=\"audio-11-1\" preload=\"none\" style=\"width: 100%;\" controls=\"controls\"><source type=\"audio\/mpeg\" src=\"https:\/\/podcasts.la.utexas.edu\/the-connector\/podcast-player\/11\/ep-04-polling-roundtable-pt-1.mp3?_=1\" \/><a href=\"https:\/\/podcasts.la.utexas.edu\/the-connector\/podcast-player\/11\/ep-04-polling-roundtable-pt-1.mp3\">https:\/\/podcasts.la.utexas.edu\/the-connector\/podcast-player\/11\/ep-04-polling-roundtable-pt-1.mp3<\/a><\/audio>","episode_data":{"playerMode":"dark","subscribeUrls":[],"rssFeedUrl":"https:\/\/podcasts.la.utexas.edu\/the-connector\/feed\/podcast\/the-connector","embedCode":"<blockquote class=\"wp-embedded-content\" data-secret=\"3IAt3fPkuG\"><a href=\"https:\/\/podcasts.la.utexas.edu\/the-connector\/podcast\/ep-04-polling-roundtable-pt-1\/\">Ep 04 &#8211; Polling Roundtable Pt 1<\/a><\/blockquote><iframe sandbox=\"allow-scripts\" security=\"restricted\" src=\"https:\/\/podcasts.la.utexas.edu\/the-connector\/podcast\/ep-04-polling-roundtable-pt-1\/embed\/#?secret=3IAt3fPkuG\" width=\"500\" height=\"350\" title=\"&#8220;Ep 04 &#8211; Polling Roundtable Pt 1&#8221; &#8212; The Connector\" data-secret=\"3IAt3fPkuG\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\" class=\"wp-embedded-content\"><\/iframe><script type=\"text\/javascript\">\n\/* <![CDATA[ *\/\n\/*! This file is auto-generated *\/\n!function(d,l){\"use strict\";l.querySelector&&d.addEventListener&&\"undefined\"!=typeof URL&&(d.wp=d.wp||{},d.wp.receiveEmbedMessage||(d.wp.receiveEmbedMessage=function(e){var t=e.data;if((t||t.secret||t.message||t.value)&&!\/[^a-zA-Z0-9]\/.test(t.secret)){for(var s,r,n,a=l.querySelectorAll('iframe[data-secret=\"'+t.secret+'\"]'),o=l.querySelectorAll('blockquote[data-secret=\"'+t.secret+'\"]'),c=new RegExp(\"^https?:$\",\"i\"),i=0;i<o.length;i++)o[i].style.display=\"none\";for(i=0;i<a.length;i++)s=a[i],e.source===s.contentWindow&&(s.removeAttribute(\"style\"),\"height\"===t.message?(1e3<(r=parseInt(t.value,10))?r=1e3:~~r<200&&(r=200),s.height=r):\"link\"===t.message&&(r=new URL(s.getAttribute(\"src\")),n=new URL(t.value),c.test(n.protocol))&&n.host===r.host&&l.activeElement===s&&(d.top.location.href=t.value))}},d.addEventListener(\"message\",d.wp.receiveEmbedMessage,!1),l.addEventListener(\"DOMContentLoaded\",function(){for(var e,t,s=l.querySelectorAll(\"iframe.wp-embedded-content\"),r=0;r<s.length;r++)(t=(e=s[r]).getAttribute(\"data-secret\"))||(t=Math.random().toString(36).substring(2,12),e.src+=\"#?secret=\"+t,e.setAttribute(\"data-secret\",t)),e.contentWindow.postMessage({message:\"ready\",secret:t},\"*\")},!1)))}(window,document);\n\/\/# sourceURL=https:\/\/podcasts.la.utexas.edu\/the-connector\/wp-includes\/js\/wp-embed.min.js\n\/* ]]> *\/\n<\/script>\n"},"_links":{"self":[{"href":"https:\/\/podcasts.la.utexas.edu\/the-connector\/wp-json\/wp\/v2\/podcast\/11","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/podcasts.la.utexas.edu\/the-connector\/wp-json\/wp\/v2\/podcast"}],"about":[{"href":"https:\/\/podcasts.la.utexas.edu\/the-connector\/wp-json\/wp\/v2\/types\/podcast"}],"author":[{"embeddable":true,"href":"https:\/\/podcasts.la.utexas.edu\/the-connector\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/podcasts.la.utexas.edu\/the-connector\/wp-json\/wp\/v2\/comments?post=11"}],"wp:attachment":[{"href":"https:\/\/podcasts.la.utexas.edu\/the-connector\/wp-json\/wp\/v2\/media?parent=11"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/podcasts.la.utexas.edu\/the-connector\/wp-json\/wp\/v2\/tags?post=11"},{"taxonomy":"series","embeddable":true,"href":"https:\/\/podcasts.la.utexas.edu\/the-connector\/wp-json\/wp\/v2\/series?post=11"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}