Political Contributions

<!-- Styles for R syntax highlighter

Working with the FEC Political Contribution Data

options(encoding = "native.enc", replace.assign = TRUE, width = 90)

The Problem

I recently read a blog that announced the Forest of Advocacy website. In their words,

This site is dedicated to providing visualizations gleaned from big data regarding politics. We will provide at least one new visualization every week from now to the election. So enjoy, and come back next Monday.

The first visualization they did examined the political contraptions of individuals over time within particular organizations. From this, as they point out,

Over the decades, one sees lines sketched out, reflecting the partisanship of individuals over time. For each organization, we also provide the net contributions of the entire organization, and the names of biggest Democratic, Republican, and 'bipartisan' contributors (the individual with the highest product of Democratic and Republican contributions).“

This made me ask the question what about the University of Minnesota? To which party do their employees contribute?

Obtaining the Data

As a data geek, I immediately went to the Federal Election Commission webpage and saw what I could download to answer this question. On their Downloadable Data Files page there were several files that I could download. As with most Federal government datasets, it is not intuitively obvious what data these files contain given their names. Unlike most datasets, these have detailed codebooks to accompany each dataset! (Props to you FEC.)
I wanted the individual contributions data (Detailed Data File). The FEC has individual contribution data for every year back to 1999–2000 available for download. I downloaded the ind12.zip data (unzips to a 327.7 MB .txt file) which has 1,994,874 records of contribution(s) from an individual to a federal committee if the contribution was at least $200. I also grabbed the Committee Master File, the Candidate Master File, and the Candidate Committee Linkage File. These files contain information that can be merged with the contributions data to link the committee and candidate information with that of the contributor.

Importing the Data into R

Now comes the fun…getting this into R so that we can actually get the data we need to answer our question. Let's start with the individual contributions data. This is a text file with 21 variables that are delimited by the pipe (|) character. The first line of data looks like this:

C00492371|N|M2|P|11990223949|15|IND|BIRDWELL, BRAD A.|CYPRESS|TX|77429|BIRDWELL|
CONTRACTOR|01232011|500||SA17A.4153|714660|||4021720111136021419

The natural choice to read in such data is to use the read.table() function. So I tried it.

pc <- read.table(file = "~/Desktop/itcont.txt", header = FALSE, sep = "|")

## Error: line 37 did not have 21 elements

For some reason the function believed that line 37 didn't have the 21 variables that the previous lines did. What to do? You could open the file in a text editor and see if there is a problem with line 37. However, my thought was, if there is an error at line 37 that I have to fix and there are 1,994,874 records this could be an all-year process of finding and fixing problems.
My solution was to use a little known function called count.fields(). This function counts the number of fields, as separated by sep=, in each of the lines of the file to be read. Then I used the table() function to figure out how many problematic lines there were.

n <- count.fields(file = "~/Desktop/itcont.txt", sep = "|")
table(n)

## n
##       8       9      12      13      16      17      18      20      21 
##    8486     290   16700    1673      23     943      14     685 1966060

It looks like there are several lines (n = 28,814) that do not seem to have 21 fields. The largest set seems to have only 12 field. Below I use the readLines() function to read in the data line-by-line, and in the next line, I identify the lines that only seemed to have 12 fields. (Note: I use the head() function to save space.)

x <- readLines("~/Desktop/itcont.txt")
head(x[n == 12])

## [1] "C00063164|N|M2||11930387788|15|IND|GOMPELS, NINA|SAVANNAH|GA|31404|MCDONALD'S|LICENSEE|01312011|250||2A1FCCF1182FA65F0CC|714744|||4031620111137409329"         
## [2] "C00063164|N|M2||11930387788|15|IND|GRUBER, GREG|SANDY|UT|84093|MCDONALD'S|LICENSEE|01312011|500||8C3DFE145D4B7356E88|714744|||4031620111137409330"             
## [3] "C00063164|N|M2||11930387788|15|IND|GUERRERO, TILA|WATSONVILLE|CA|95076|MCDONALD'S|LICENSEE|01312011|1000||D5C6C704081D8AD1910|714744|||4031620111137409331"    
## [4] "C00063164|N|M2||11930387779|15|IND|ABBATE, JAMES A.|MERCED|CA|95341|MCDONALD'S|LICENSEE|01312011|2000||0ECDDCDE3B4CF443D22|714744|||4031620111137409302"       
## [5] "C00063164|N|M2||11930387779|15|IND|ADAMS, ALLISON|TAMPA|FL|33607|MCDONALD'S|LICENSEE|01312011|500||543088A9C13C4BE1A71|714744|||4031620111137409304"           
## [6] "C00063164|N|M2||11930387780|15|IND|ARMSTRONG, DONALD D.|BEAVERTON|OR|97006|MCDONALD'S|LICENSEE|01312011|2600||8DBDD6D125EDFAB256C|714744|||4031620111137409307"

Each of these lines stopped reading 12 fields in. The problem with these lines is the apostrophe in the 12th field. R is treating this like a string, so everything after the apostrophe appears as a single field (there is no closing apostrophe). We can remedy this by using the quote=argument to let R know that strings need to be in double-quotes rather than single quotes. One more issue is that we cannot write quote=""", because R will usethe first two quotation marks. We need to escape the middle quote using the backslash (\). Now we try again,

pc <- read.table(file = "~/Desktop/itcont.txt", header = FALSE, sep = "|", quote = "\"")

## Error: line 1151 did not have 21 elements

Okay. This time we made it through 1,150 lines before the code broke. Again we use count.fields() this time adding the quote= argument.

n <- count.fields(file = "~/Desktop/itcont.txt", sep = "|", quote = "\"")
table(n)

## n
##       8       9      12      13      20      21 
##      13       1     439      18      73 1994330

Once again, the 12th field is giving problems. We again examine some of those fields.

head(x[n == 12])

## [1] "C00030718|N|M3||11930496503|15|IND|DURANTE, KATHLEEN S|CHEYENNE|WY|82009|#1 PROPERTIES|REAL ESTATE BROKER|01262011|300||A09703181A2EF43EFA3B|717445|||4031820111137607355"    
## [2] "C00030718|N|M3||11930496503|15|IND|BIGGS, CYNTHIA L MRS.|LARAMIE|WY|82070|#1 PROPERTIES|REAL ESTATE BROKER|01262011|300||A553A97B481D44B5EA16|717445|||4031820111137607357"   
## [3] "C00401190|N|Q1|P|11930643549|15|IND|PEER, WILBUR T.|UPPER MARLBORO|MD|20774|MANAGEMENT CONSULTANTS INC #1|CONSULTANT|03312011|500||C4754843|722315|||4041320111138070572"     
## [4] "C00089086|N|M2||11990218895|15|IND|MONTGOMERY, SHERI|EVANSTON|WY|82930|UNITA COUNTY SCHOOL DIST #4|OCCUPATIONAL THERAPIST|01192011|400||38594231|713984|||4031120111137183347"
## [5] "C00349746|N|M2||11990229557|15|IND|ANKER, ED|VISALIA|CA|93291|SHADY ACRES DAIRY #2|DAIRYMAN|01122011|261||SA11AI.28076|715177|||4031520111137265488"                          
## [6] "C00349746|N|M2||11990229568|15|IND|ROELOFFS, JOHNNY|TIPTON|CA|93272|J R DAIRY #2|DAIRYMAN|01122011|292||SA11AI.28030|715177|||4031520111137265522"

A-ha! The hash tag (pound sign) is causing problems. Remember that R treats the hash tag as a character to indicate a comment. Thus, everything on a line following a hash tag is commented out. We want the hash tag to be part of the string in that field. We can now add the argument comment.char="" to the read.table() function.

pc <- read.table(file = "~/Desktop/itcont.txt", header = FALSE, sep = "|", quote = "\"", comment.char = "")

Great! It worked! Well…we should check that.

nrow(pc)

## [1] 1994874

length(pc)

## [1] 21

Perfect. Now we can use the online codebook to add the variable names to our data.

names(pc) <- c("CMTE_ID", "AMNDT_IND", "RPT_TP", "TRANSACTION_PGI", "IMAGE_NUM", "TRANSACTION_TP", 
    "ENTITY_TP", "NAME", "CITY", "STATE", "ZIP_CODE", "EMPLOYER", "OCCUPATION", "TRANSACTION_DT", 
    "TRANSACTION_AMT", "OTHER_ID", "TRAN_ID", "FILE_NUM", "MEMO_CD", "MEMO_TEXT", "SUB_ID")

Reading in the Other Files and Merging the Results

While we can breathe a big sigh of relief at getting the contribution data read in, it isn't enough. A quick scan of the variables suggest that this file does not include important information. For example, which political party the individual contributed to. Scanning the codebooks for the other files—the Committee Master File, the Candidate Master File, and the Candidate Committee Linkage File—we find that both the Committee Master File and the Candidate Master File have party affiliation data.
How do you get the data from these files into our individual contributions data? To do this we use the merge() function. Before doing that, however, it is worth a quick diversion to talk about databases and tables.

Databases

Databases are alternative ways to store data. They are particularly useful for large datasets like this one. The key to understanding databases is to remember that they are composed of tables. Tables are nothing more than rectangular arrays (they have rows and columns). For example, in R, a data frame is a table—it is made up of rows and columns. Most databases include multiple tables.
We can think of each of the files from our political contribution files as tables.

File	Rows x Columns
Individual Contributions File	1,994,874 x 21
Committee Master File	13,093 x 15
Candidate Master File	5,345 x 15
Linkage File	5,954 x 7

Note that from the codebooks we can also see that these "tables” have some columns with common variable names, other variables are different across tables. This is because in a database, each table (should) represent different information. The common variable names are used to link information across tables. For example, the Individual Contributions File contains file on individuals who made contributions and the Committee Master File contains information on political committees. The variable CMTE_ID is common to both tables. So, we can use that to link the two tables or files.

Merging Data Frames

To examine how we will merge two data frames together, we first read in the Committee Master File and Candidate Master File.

cm <- read.table(file = "~/Desktop/cm.txt", sep = "|", header = FALSE, quote = "\"", comment.char = "")
nrow(cm)

## [1] 13093

length(cm)

## [1] 15

names(cm) <- c("CMTE_ID", "CMTE_NM", "TRES_NM", "CMTE_ST1", "CMTE_ST2", "CMTE_CITY", "CMTE_ST", 
    "CMTE_ZIP", "CMTE_DSGN", "CMTE_TP", "CMTE_PTY_AFFILIATION", "CMTE_FILING_FREQ", "ORG_TP", 
    "CONNECTED_ORG_NM", "CAND_ID")
cn <- read.table(file = "~/Desktop/cn.txt", sep = "|", header = FALSE, quote = "\"", comment.char = "")
nrow(cn)

## [1] 5345

length(cn)

## [1] 15

names(cn) <- c("CAND_ID", "CAND_NAME", "CAND_PTY_AFFILIATION", "CAND_ELECTION_YR", "CAND_OFFICE_ST", 
    "CAND_OFFICE", "CAND_OFFICE_DISTRICT", "CAND_ICI", "CAND_STATUS", "CAND_PCC", "CAND_ST1", 
    "CAND_ST2", "CAND_CITY", "CAND_ST", "CAND_ZIP")

We then will use the merge() function to join the two tables (data frames) together. This function takes as its initial two arguments the names of the objects that the data frames are assigned to, namely x= and y= (e.g., x=pc and y=cm). It is important to note that the data in the second argument (y) will be added to the rows of data from the first argument (x). After providing the names of the data frames, the argument by= is used to give the names of the variable names that are common to both data frames. (If there is more than one, use the c() function to bind them together.) We also use the all.x=TRUE argument. This will keep any rows in x that don't have a match in y. If this argument is not included then rows in x without matches in y will be dropped.

pccm <- merge(x = pc, y = cm, by = "CMTE_ID", all.x = TRUE)
nrow(pccm)

## [1] 1994874

length(pccm)

## [1] 35

The new data frame will have the same number of rows as the data frame in the x= argument. The number of variables will be equal to the sum of the unique variables in both data frames and the number of common variables between the two. Note how the results are different if we reverse the order of the two data frames.

cmpc <- merge(x = cm, y = pc, by = "CMTE_ID", all.x = TRUE)
nrow(cmpc)

## [1] 2001639

length(cmpc)

## [1] 35

The number of variables is the same, but the number of rows is different because this time the information from the pc data frame was added to the rows in the cm data. In SQL terms, this is called a left-inner join. If you are familiar with SQL, you might like to use the sqldf package.
We can also join the candidate data to our pccm data.

pccmcn <- merge(x = pccm, y = cn, by = "CAND_ID", all.x = TRUE)
nrow(pccmcn)

## [1] 1994874

length(pccmcn)

## [1] 49

Finding University of Minnesota Contributors with Grep

The data frame includes data from all contribotors, and I was interested in only the University of Minnesota employees. The variable EMPLOYER contains information on the contributor's employer. This variable contains character (string) data. To find particular strings we can use the grep() function. Grep was named after the Unix qed/ed editor subcommand “g/re/p”. In this subcommand, g stands for global, re stands for a regular expression, and p stands for print. In essence these subcommands would globally search some text for a given regular expression and print the lines containing matches.
The grep() function takes as its first argument, the character string to search for and as its second argument the variable (or vector to search). Here we search for the string “UNIVERSITY OF MINNESOTA” in the EMPLOYER variable and print the first fe results.

head(grep("UNIVERSITY OF MINNESOTA", pccmcn$EMPLOYER))

## [1] 13529 14294 19110 21758 22403 23838

The function outputs the row number for any row in which the string “UNIVERSITY OF MINNESOTA” is found in the EMPLOYER variable. We can use indexing to assign the rows of data that include this string in a new data frame.

umn <- pccmcn[grep("UNIVERSITY OF MINNESOTA", pccmcn$EMPLOYER), ]
nrow(umn)

## [1] 336

length(umn)

## [1] 49

The output shows that 336 rows have the string “UNIVERSITY OF MINNESOTA” as the employer. Note also that the new data frame has retained all of the columns as our previously merged data frame. We can now examine the affilation of the political party that these emplyees contributed to.

table(umn$CAND_PTY_AFFILIATION)

## 
##     AIP AMP CIT CON CRV CST DCG DEM DFL FED GRE GRN IAP IDP IND  IP LIB   N N/A NNE NOP 
##   0   0   0   0   0   0   0   0 165  27   0   1   0   0   0   0   0   0   0   0   0   0 
## NPA OTH PAF PRI REF REP RTL SUS SWP TEA  TX UNK   W WFP 
##   0   0   0   0   0  42   0   0   0   0   0   0   0   0

table(umn$CMTE_PTY_AFFILIATION)

## 
##       0 ACP AIP AMP CIT COM CON CRV CST DEM DFL FED FWP GOP GRE GRN HOS IND LIB  MW NNE 
##  78   0   0   0   0   0   0   0   0   0 150  63   0   0   0   1   0   0   0   0   0   0 
## NON NPA OTH PAC PAF PRO REF REP SOC STA SUS SWP TEA UNK UPA UTP  WF WFP WHG WNC 
##   0   0   0   0   0   0   0  43   0   0   0   0   0   1   0   0   0   0   0   0

The only parties that University of Minnesota employees have contributed to are the Democratic, DFL, and Republican. We will append a variable to the umn data frame called PARTY that has two levels DEM/DFL and REP.

umn$PARTY[umn$CAND_PTY_AFFILIATION == "DEM"] <- "DEM/DFL"
umn$PARTY[umn$CAND_PTY_AFFILIATION == "DFL"] <- "DEM/DFL"
umn$PARTY[umn$CMTE_PTY_AFFILIATION == "DEM"] <- "DEM/DFL"
umn$PARTY[umn$CMTE_PTY_AFFILIATION == "DFL"] <- "DEM/DFL"
umn$PARTY[umn$CAND_PTY_AFFILIATION == "REP"] <- "REP"
umn$PARTY[umn$CMTE_PTY_AFFILIATION == "REP"] <- "REP"
table(umn$PARTY)

## 
## DEM/DFL     REP 
##     242      44

Plotting the Contributions with ggplot

First we load the ggplot2 library.

library(ggplot2)

We plot the contributions conditional on party. (To learn more about the use of ggplot see the excellent online documentation.)

ggplot(data = na.omit(umn), aes(x = PARTY, y = TRANSACTION_AMT, color = PARTY)) +
  geom_jitter(alpha = 0.3, position = position_jitter(width = 0.1, height = 0)) +
    scale_color_manual(name = "Affiliation", values = c("Black", "Red")) +
    xlab("") +
    ylab("Contribution") +
    theme_bw()

We note that four observations are negative.

umn2 <- umn[umn$TRANSACTION_AMT > 0, ]
ggplot(data = na.omit(umn2), aes(x = PARTY, y = TRANSACTION_AMT, color = PARTY)) +
  geom_boxplot() +
  geom_jitter(alpha = 0.3, position = position_jitter(width = 0.1, height = 0)) +
  stat_summary(fun.y = "mean", geom = "point", size = 4) +
  scale_color_manual(name = "Affiliation", values = c("Black", "Red")) +
    xlab("") +
    ylab("Contribution") +
    theme_bw()

Removing these, we observe that the political contributions seem to be comparable (medians are roughly equal) for the two sides of the political aisle. However, there is a great deal more variation in the Republican contributions.
--> Type MIME Type text/html Resource Type Document Location Full URL http://rstudio-pubs-static.s3.amazonaws.com/1689_9eee1ef2fbe948f6ba5901a17ca52308.html Scheme http Host rstudio-pubs-static.s3.amazonaws.com Path /1689_9eee1ef2fbe948f6ba5901a17ca52308.html Filename 1689_9eee1ef2fbe948f6ba5901a17ca52308.html

How Google Makes its Maps

An exciting piece of journalism about data and computing appeared in the Atlantic.

"Behind every Google Map, there is a much more complex map that's the key to your queries but hidden from your view. The deep map contains the logic of places: their no-left-turns and freeway on-ramps, speed limits and traffic conditions. This is the data that you're drawing from when you ask Google to navigate you from point A to point B -- and last week, Google showed me the internal map and demonstrated how it was built. It's the first time the company has let anyone watch how the project it calls GT, or "Ground Truth," actually works. " Read the article online here [read article].

Heat Map of Score Difference for Big Ten Teams (week 2)

2–0

That is the record of the Gophers. It has been a long time since that has occurred. It feels good. I finally know what it is like to feel like a USC or LSU fan. To even make the weekend better, the Vikings also won, and the Badgers and Packers lost! When is the last time we had a Gopher/Viking win and Badger/Packer loss in the same weekend???

On a more somber note, my condolences to Iowa fans. Your neighbors to the north know exactly what it feel like to lose to a team you should beat, and in that loss realize that you need a running back, a quarterback, a defense, a wide receiver, and a secondary. We know the agony of three quarters of hope followed by a quarter of disaster and pain–seeing your season turn from a bowl-bound 7-5 to 2-10 on a single interception or failed first down. Been there.

I read a few excellent articles and blog postings over the weekend. One such blog entry gave a great answer to the question that is asked to all faculty members at one point, aside from teaching, what do academics do? [Read the blog here.] The author, a USC faculty member, has categorized our work into five dimensions that nicely summarize what we do:

• Learning
• Discovering
• Sharing
• Helping 
• Fund-raising

These dimensions, as he points out, are waaaay more useful then teaching, research and service, the big-three that universities seem to have coveted for years. The last two (helping and fund-raising) I would like to see changed, not because they are incorrect, but rather because they don't have the "humph" that the others do. They are quite descriptive, and encompass what we do, but you would never see the universities starting a campaign of "Driven to Help" or "Driven to Fund-Raise". Maybe citizenship would be better than helping. But, I do like the participle "ing" on the end of each noun....and citizening doesn't cut it.

Another awesome read last night was Jay-Z’s “99 Problems,” Verse 2: A Close Reading With Fourth Amendment Guidance for Cops and Perps. This is a line-by-line analysis of the second verse of Jay-Z's masterpiece written from the perspective of a criminal procedure professor. He examines the ins-and-outs of various case law relating to the legalities of a traffic stop as documented by Mr. Z. For example, he writes

A traffic stop is a legitimate seizure of the person, for purposes of investigating the violation of the traffic law and writing up the citation. But it cannot be prolonged for longer than reasonably necessary to complete that legitimate activity...And the Fourth Amendment rule is very clear: if the police detain you after they’ve finished processing the ticket—or if they simply dawdle over the ticket processing for an unreasonable length of time—in order to get a K-9 team there, then the eventual dog sniff will be the fruit of an illegal detention, and any evidence found will be suppressed. The officer in Jay-Z’s case apparently knew this, and so released Jay-Z after the stop when the K-9 unit he’d called was late in arriving. I got 99 problems but a bitch ain’t one...

Lastly, I have been reading the Nieman Journalism Labs review of J-schools. They have had big-wigs comment on what these schools are doing well and what they wish students would have learned. The most common wish has been that journalists learn some coding. Here are two takes.

Brian Boyer, the head of NPR’s news apps team, suggests a course in Hacker Journalism 101. He provides a reading list and some assignments. He also offers three virtues of a programmer that the students would apply to all of their work in the course,

• Laziness: I will do anything to work less.
• Impatience: The waiting, it makes me crazy.
• Hubris: I can make this computer do anything.

Miranda Mulligan, executive director at Northwestern’s Knight News Innovation Lab, also promoted learning to code, arguing that journalists need to learn how to code if they want to become better (and employable) storytellers. [Read Miranda's take here.]

And yet if you attend any event with a collection of jouro-nerd types, inevitably the same question will come up. Someone will ask...“How can we tell better stories on the web?” and proceed to bemoan the tedium of reading a daily newspaper and a newspaper website...It is our job as educators to remove fear of learning, a fear notoriously prevalent in journalists...For me, there’s only one response to this: Journalists should learn more about code. Understanding our medium makes us better storytellers. For an industry that prides itself on being smart, tolerating ignorance of the Internet is just stupid.

Forgot one other thing. Saw this and thought every undergraduate should be given an orientation to the information on searching with Google in these slides.

Herbie Goes Gopherrific

Herbie is the name of my MINI Cooper. The MINI Cooper that I had a love-hate (mostly hate) relationship with until recently. About two years ago, I traded by Audi TT Quattro for the MINI. I didn't love the Audi, but it was a convertible and was very fast. All in all, it was a fun car to drive. I liked the look of the MINI when I got it, but in retrospect, it hasn't been fun to drive (an essential characteristic of any car in my book). I made two huge mistakes: I got an automatic transmission and I didn't get the S.

The S(upercharged) MINI should be the only Cooper allowed by Federal law in any state in which freeway driving is a must. The only thing that has saved my life on freeway entry with Herbie is that mostly older folks and Minnesotans frequent the freeway. Herbie has no acceleration, an S would have helped.

Manual transmissions are just more fun to drive. Automatics suck. 'nuff said about that. These lacks in Herbie have had me recently (for the last year and 1/2) scouring eBay for a new car. I was ready to sell off Herbie and move on. Tim, however, talked me into keeping Herbie and also buying a new car. This, required that I drive Herbie for one more year, before I get something more fun to drive. So, how do you transform a car you hate into something that you can tolerate driving for another year?

You make it into a living tribute to your alma-mater. I started by heading to Red Rooster with my CEHD University of Minnesota t-shirt. After several books of paint samples, we were able to match the maroon pretty closely. Then Tim came over and we had a one-day paint-a-thon in our garage–sanding, bond, more sanding, more bondo, priming, painting, and clear coat.

I also got some Gopher swag to jazz up the MINI. I replaced the old checkerboard on the roof with Goldy. I also got some chrome University of Minnesota emblems to replace the MINI emblems. I replaced the C-O-O-P-E-R chrome lettering on the trunk with G-O-P-H-E-R. Lastly, I got some vinyl yellow/gold stripes and rocker panels to finish out the car.


Although I am still dreaming of a Porsche Targa, I can at least tolerate Herbie again.

One of My Favorite House Gifts

Pavel and Patty are two very cool people (despite the fact that they cheer for the Bears)! P&P are two of the people learning Open-C at Lauren's training class. This last week, they came over for drinks and dinner, as well as to meet the dogs. They also came bearing gifts.

A toaster and a very large Justin Bieber card which sings when you open it.

I got a downstairs toaster! And props to P&P...they know their toasters. They didn't bring one of those gaudy nouveaux four-slicers. They went with the two-slicer! Serious toast architects understand that while the four-slicer may be appealing on volume of toast, that the two-slicer produces a better overall consumption experience. The butter-melt is perfect on each slice (try that with a four-slicer...by the time you are buttering that last slice it has cooled to near frost-bite like temperature). And nobody wants to wait through the preparation of four slices before eating. That would be crazy. But, with the four-slicer, wait you will, because as a toast optimist there is always the hope of melted butter on even that last slice, and that hope is enough to keep on preparing through the last slice before even a morsel of the sweet nectar-like ambrosia of any toast is indulged upon.

Sleek, sophisticated and designed for excellent toast

This toaster has seven toast shade settings to "allow for customized toasting". Now, in my opinion, this is six more than is necessary, but it nice to have the option. The extra-wide and longer slots accommodate a variety of foods for toasting including bagels (there is also a bagel option button), texas toast, english muffins, and grilled cheese (just turn the toaster horizontally) without the friction rub that accompanies food toasted in a smaller slot.

There is also a cancel button which immediately shuts down the toasting cycle and elevates the food using the near perfect toast lift. This toaster even features a warm button, which the box claims, "warms without further toasting". This is a big waste on a two-slice toaster. Who isn't going to get to their second slice? Any person who would need a toast warmer on a two-slice toaster, is a person who needs all of their toasting privileges revoked. They probably also read Reader's Digest Condensed Books. Ugh. They should all be re-located to St. Paul.

The blurb on the box. I like "Toast just the way you LIKT it".

Now, no blog on such a divine food would be complete without a description or suggestion for an accompanying beverage. When serving toast, the pairing can be tricky. After all, are you topping the toast with jelly? honey? nutella? peanut butter? nothing? butter and salt? caramel sauce? Or is it the base of the perfect sandwich? (I won't even cover when idiots use toast as a side, that is just nonsense.)

P&P brought over what may be regarded by most toast experts to be the perfect pairing for toast. RumChata and Three Olives Loopy Vodka. Mix together and add (if desired) a dash of heavy cream. Wallah! (kiss thumb and forefinger) Perfection. Hints of Cinnamon Toast Crunch with a nose of Fruit Loops. Chateau d'Yquem my foot. This is heaven in a glass. It readies the palate for toast of any kind.

Why American Schools Stink

An interesting take in Bloomberg Business Week on why the education system in the United States is not up to par. Their conclusion is that it is because of the parents of American students, or rather the lack of commitment to education that their parents display. [Read the article here.] Some of the causes they shoot down, and the data they use to do it, are

• Students are slackers: Between 2002 and 2009, the U.S. high school graduation rate climbed three percentage points, so that more than three quarters of all students now get a diploma.  And the average school kid is learning more than ever before (United States Department of Education’s National Assessment of Educational Progress).
• Teachers are incompetent and don't work hard: Less than eight percent of teachers in their survey ranked below “basic” competence and the average teacher may be working an eleven-hour day (Measuring Effective Teaching: A Potential for Change, Bill and Melinda Gates Foundation)
• American schools suffer from a lack of resources: The average U.S. student costs around $80,000 to educate from the age of six to fifteen.  Only Switzerland spends at a similar level, and the Czech Republic, which scores higher that the U.S. on the international math tests, spends about a third of that amount (The Economics of International Differences in Educational Achievement, Stanford University).

After ruling these causes things out, they settle on parents as the problem. As with so many other issues, the problem is much more complex and there is no easy catch-all solution, even though the news and many politicians wish there was.

The two things that really caught my eye (and I agree with...duh...I'm blogging about it) were

1. The calling out of the legislative nonsense that No Child Left Behind has become. I especially like that the article acknowledges that we can have standards and benchmarks (and should) yet needn't mandate standardized tests to have these. 
2. The suggestion that the culture needs to shift in order to change anything. The unfortunate part of this is that the article ends all touchy-feely about it. Rah-rah parents. You can make a difference! I guess this crap sells magazines/journals. 

Last Thoughts: The article reports, "Around the world, the catch-all measure used to proxy for parental commitment to education is the number of books in a child‘s household".  Proxies are necessary for capturing so many latent traits and variables that their use is ubiquitous in every educational study. One question...in the technology age, especially that of e-books, will educational researchers continue to measure this proxy?  Should they? 

Two Scholarships I Wish I Could Have Applied For

Zombie Apocalypse Scholarship
It’s not so much a question of if the zombie apocalypse happens, but more of when it happens. And if you have ever watched the Walking Dead, you sometimes ponder escape strategies in a variety of situations. Well, why not get money for that?

ScholarshipExperts.com is offering college students a $1,000 scholarship for writing the best zombie-escape plan. In the essay, you must answer where you would run in the event of a zombie outbreak at your school and what five items you would take to ensure survival. It must not exceed 250 words. The deadline is Nov. 30.

Jon Fortenbury can get your creative juices flowing when you read his article about choosing a zombie-proof campus.



Stuck at Prom
If your inner-punk-rocker never died, then you’ll love this scholarship, since it asks you to rebel at what’s allegedly one of the most important events of your life: prom.

The Stuck at Prom Scholarship Contest (sponsored by Duck Brand Tape....get it...."Stuck"!) requires you and your date to accessorize those prom outfits with duck tape and wear it to the dance. The couple with the best outfit wins $5,000. Second place wins $3,000 and third place wins $2,000. The seven remaining runner-ups still score $500. Deadlines are in June each year.

Note to Mandy: We could have killed this!

Who Gets the Credit?

I read this Op-Ed piece the other day and was reminded again how much I love David Brooks. A reader wrote the following question

Over the past few years, I've built a successful business. I've worked hard, and I'm proud of what I've done. But now President Obama tells me that social and political forces helped build that. Mitt Romney went to Israel and said cultural forces explain the differences in the wealth of nations. I'm confused. How much of my success is me, and how much of my success comes from forces outside of me?

Brooks' reply is not only eloquent, but very funny [read it here]. Two things that particularly resonate are that in the opening line he makes mention of multiple regression, and the reference to Ayn Rand also made me laugh. (Note to Greg: Please pay attention to the age-group that buys into her crap.) 

Lastly, I believe the last paragraph of the essay should be required reading (and should probably be memorized) for every business school student from here on.

Great companies, charities and nations were built by groups of individuals who each vastly overestimated their own autonomy. As an ambitious executive, it's important that you believe that you will deserve credit for everything you achieve. As a human being, it's important for you to know that's nonsense.

Some Miscellany to Catalog

In a blog post I read recently, a Carnegie Mellon statistics professor was waxing-on about the differences between a "statistician" and a "data scientist" [read blog].  He writes,

If people want to call those who do such jobs "data scientists" rather than "statisticians" because it sounds more dignified, or gets them more money, or makes them easier to hire, then more power to them. If they want to avoid the suggestion that you need a statistics degree to do this work, they have a point but it seems a clumsy way to make it. If, however, the name "statistician" is avoided because that connotes not a powerful discipline which transforms profound ideas about learning from experience into practical tools, but rather, a meaningless conglomeration of rituals better conducted with twenty-sided dice, then we as a profession have failed ourselves and, more importantly, the public, and the blame lies with us. Since what we have to offer is really quite wonderful, we should not let that happen.

The italicized bit is mine. I think it sums up what myself and others have felt for awhile now. It may seem that what we call ourselves is irrelevant, but maybe it isn't. Many companies or products go through a re-branding process to improve their image or make themselves sexier. The cynical side of me says that this is just an effort to increase revenue, but the almost ever present flip-side is that it works. As a discipline, we are competing with other disciplines for the revenue of people. Currently, students have more choices than ever before in the history of education. Re-branding the profession may be exactly what is needed.

However, changing the nomenclature or title (even the name of the degree) will not be enough. Too many statistics courses are still taught in a manner in which students come out feeling that the discipline is a meaningless conglomeration of rituals better conducted with twenty-sided dice. The content in the introductory course hasn't changed (in many courses) in twenty years. I am positive that some people who read that last sentence believe that to be a good thing. They likely view education through nostalgic lenses.

Several years ago, psychology researchers published an article in Journal of Personality and Social Psychology in which they identified the content, triggers, and functions of nostalgia. One key finding was that people seem to engage in nostalgia specifically to make themselves feel better, which suggested that we may be unconsciously biased towards remembering things that make us happy and against remembering the things that do not. Human beings have a remarkable propensity towards this bias, requiring far less information to confirm beliefs when they are consistent with our current state of mind. In the psychology literature, this is known as “confirmation bias”, and there is a substantial body of research that has shown that people are predisposed to remember more of the good things in life.

My experience (personal data) was that I did not understand the true nature, power, and sexiness of statistics (or data science) until my Ph.D. work. Some of this is, of course, related to being able to see the forest for the trees, but much of it was the problems that were presented in my earlier course work. Seeing a disciple as one that transforms profound ideas about learning from experience into practical tools requires profound questions and problems. The fact is that many students make decisions about their future course work and major after a single course. One great experience in an early course is all that is needed to forever tie that course into our memories. A terrible experience forever associates the discipline with negative feelings. The first course is the most important one for the discipline. We have an obligation to do better by our students in this first course. If that includes re-branding ourselves as data scientists, so be it.



On a lighter note, my favorite quote about R appeared in a NYT article about bond traders.

The traders here are mostly educated in math or physics, often outside the United States, and their desks are piled high with textbooks like the “R Graphs Cookbook,” for working with obscure computer programming languages.

You can read the article here [read article].

Where has Summer Gone?

I cannot believe that the Joint Statistics Meetings are upon us again. Once again, this summer seems to have flown by and I haven't even come close to getting the things done that I wanted to. Just a quick update (more to communicate this in the public arena for my own accountability) on what I have been doing and what I need to yet finish.

First, I taught a regression course this summer (technically we have a week to go). I updated all of my notes to use ggplot2 in the course. For myself, I also updated all of them using knitr, which I finally have compiling in TexShop. The PDF format of all the notes, along with the R scripts, data, and a bevy of other info is on the course website (open to the public) at http://www.tc.umn.edu/~zief0002/8262.php.

I have also been putting our course activities, homework, etc. from the CATALST project into a book that we will use in the EPsy 3264 course. I am almost through two of the three units, but I have a bad feeling about Unit 3. It has been my nemesis for two years running, and although I am more satisfied with its current instantiation, it still doesn't blow my socks off.

Speaking of socks, I got some sweet dachshund socks last week. Unlike the photo, mine are bright yellow. I have also been working on a project with the University extension program to evaluate the education program of the Supplemental Nutrition Assistance Program. It has been fun to help them think about study design. In addition, I have been working with a student to clean and analyze data collected at the Walk-In Counseling Center. We have tens of thousands of observations from as far back as 1992! It may be rich (or it may not), unfortunately we are still on the cleaning part. I think the plan is to eventually make the data public. Stay tuned.


For fun I have been working on the landscaping, although with the heat this summer it has been slow. The stone pathway remains unfinished, but I have gotten a little further on it. We also just planted some wild prairie (I know it is the wrong time of year to seed, but so it is). I have also been casting stepping stones for the auxiliary walking path. Maybe once I finish the walkway, I will post some pics.


Tim and I also painted Herbie so I could drive him for awhile longer before I buy a different car. That was quite an adventure. He is a darker more maroony-red than he was. We also bondo-ed the dents in the front and back. I also got a huge Goldy sticker for the roof. He looks pretty sweet, but soon he will look sweeter. I have some yellow hood and trunk stripes to put on. I also have some Goldy metallic emblems to replace the old MINI ones. 


Lastly, for fun, I have been watching the video from Harvard's CS50 lectures on iTunes U. This has been a real treat and I have already put some of the programming into action. I wish my computer science professors had been as animated, hip and witty as David. I also love how many opportunities those students get to actually immediately use their skills to improve the campus and community. Awesome! 


Now, on to the list of things that I need to do yet (that haven't been already mentioned):

• I need to learn D3 at a higher level and implement it using some data. I bought the D3 book from O'Reilly, but thus far I am under-whelmed. I was hoping for a bigger book, but it is one of those slim numbers that O'Reilly seems to be pushing out.
• I would like to finish a book or two that I have been meaning to read.
• I would like to add insulation, vapor barrier, etc. to the outside walls in the garage. I think this might really help since our bedroom is directly above.
• I want to finish a course proposal for a categorical data modeling course.
• I want to update all of my webpages.
• I would like to update my 8261 course. 

Call Me Maybe?

A pretty cool video of my favorite mascot....