Categories
Money People

Obscene Math

Recovered from the Wayback Machine.

I was a double major in university, psychology and computer science. Double majors weren’t all that unusual, except that most doubles were in fields that had some class overlap, such as computer science and math. The only overlap I had in my two fields were statistics courses. I could take undergraduate and graduate level statistics classes in the psych department to meet a portion of my computer science math requirements.

There were only two of us signed up for graduate level statistics class, so the professor had us meet in his office. The statistics were so complicated, we had to use computers and software created in the days before “usability” was a criteria for all of our course work. I’ve since managed to forget most of my statistics training except for one valuable lesson: don’t trust statistics. If you’re determined, you can manipulate statistics to prove any point, regardless of how extreme.

A case in point is a New York Times op piece by two gentlemen, Michael Cox and Richard Alm, from the Federal Reserve Bank in Dallas. According to their statistics, there really aren’t two separate classes, rich and poor, in this country. In fact, the poor live a comparable lifestyle to the rich.

Income statistics, however, don’t tell the whole story of Americans’ living standards. Looking at a far more direct measure of American families’ economic status — household consumption — indicates that the gap between rich and poor is far less than most assume, and that the abstract, income-based way in which we measure the so-called poverty rate no longer applies to our society […] if we compare the incomes of the top and bottom fifths, we see a ratio of 15 to 1. If we turn to consumption, the gap declines to around 4 to 1. A similar narrowing takes place throughout all levels of income distribution. The middle 20 percent of families had incomes more than four times the bottom fifth. Yet their edge in consumption fell to about 2 to 1.

The data the authors use to perform their statistics is based on the fact that though rich people invest or bank their extra income, while poor families “magically” live beyond their means, they all “consume equally” and therefore are more equal than not.

Of course, Cox and Alm gloss over the fact that most poor people are overridden in debt, barely keeping ahead of bankruptcy in order to indulge in frivolous expenditures like medical treatment.

No, Aunt Sally has a 19 inch color TV in her mobile home while Aunt May has a 60 inch top of the line plasma TV in her pad overlooking Central Park, so there really is no difference between the two.

It’s true that the share of national income going to the richest 20 percent of households rose from 43.6 percent in 1975 to 49.6 percent in 2006, the most recent year for which the Bureau of Labor Statistics has complete data. Meanwhile, families in the lowest fifth saw their piece of the pie fall from 4.3 percent to 3.3 percent.

Income statistics, however, don’t tell the whole story of Americans’ living standards.

Speechless. I’m just…speechless…

update More from Paul Krugman and Dean Baker, especially in regards to flawed sampling forming the basis for the pretty charts.

update 2 Excellent commentary from The Big Picture, who focuses only on exposing the flaws in the statistics applied (because there’s not enough time to expose all the other flaws in the writing).

Categories
Web

Google is the new Cloverfield monster

Recovered from the Wayback Machine.

Oh, the horror! Google hijacks 404 pages!

The reality is that the new Google beta toolbar doesn’t hijack the 404 page if the site provides a 404 page or other form of web error handling. I tried the toolbar out this morning, and the only case I found where the Google toolbar provided a search page is the site matching the screenshots below, and the site given in the original post on this topic. The latter site provided a lame looking redirect back to the main page. However, other sites that redirected back to the home page for 404 errors did not have this problem, so the problem seems to be unique to this site.

If you’ve ever seen default 404 error handling, you know it’s basically useless.
[missing image]

Compare that with a page managed by the toolbar.

[missing image]

I would expect a search engine toolbar to provide useful, alternative methods of finding the content if the web site uses default error handling. However, according to Codswallop, Google steals your visitors.

Why is this “helpful” behavior bad? As well as a link to the domain root they provide a prominent search box pre-filled with search terms. The temptation is going to be to hit that search button, effectively taking away your visitor.

I would say any webmaster that doesn’t provide effective error handling pages for 404 errors doesn’t really care about losing visitors, do they?

update

Matt Cutts from Google explained that the toolbar looks for a result larger than 512 bytes. The example page is nothing more a broken HTML page, with a meta refresh and a link, all of which is less than 512 bytes. Those sites that do a direct redirect don’t, of course, return 404 to trigger the toolbar. End of story

.end update

What really surprised me about this story, though, is that if people are so quick to accuse Google of ‘evil’ behavior in an innocuous situations like this, why was the idea of Google helping to bail out Yahoo to keep the latter out of the hands of Microsoft seen as a “good” thing? I would think a search engine monopoly in the hands of Google would be potentially more evil than Google providing useful features for default 404 error handling.

This environment is confusingly inconsistent at times.

Categories
RDF Specs SVG XHTML/HTML

Our bouncing baby markup has growed up

Recovered from the Wayback Machine.

On today’s tenth anniversary of the birth of XML, Norm Walsh writes:

I joined O’Reilly on the very first day of an unprecedented two-week period during which the production department, the folks who actually turn finished manuscripts into books, was closed. The department was undergoing a two-week training period during which they would learn SGML and, henceforth, all books would be done in SGML…My job, I learned on that first day, would be to write the publishing system that would turn SGML into Troff so that sqtroff could turn it into PostScript. “SGML”, I recall thinking, “well, at least I know how to spell it.”

Ah yes. “Unix Power Tools” was formatted as SGML, the one and only book at O’Reilly I worked on that wasn’t in a Word format. I must express a partiality to my NeoOffice, though the SGML system was ideal for cross-referencing and indexing. OpenOffice ODT, or OpenDocument text, will be the most likely format for the next UPT. Just another example of the permanent/impermanence of web trends.

Norm also mentions about HTML5 possibly being the nail in this child of SGML’s coffin, but as I wrote recently, the folks behind HTML5 have solemnly assured us this specification also includes XHTML5. I’d hate to think we’re giving up on the benefits of XHTML just when they’re finally being realized by a more general audience.

Of course, I’m also fond of RDF/XML, which seems to cause others a great deal of pain, the pansies. And I’ve never hidden my SVG fandom and SVG is based in XML. I must also confess to preferring XML over JSON–you know, good enough for granddad, good enough for me. Atom rules. Or is that, Atom rocks? I’m also sure XML has squeezed between the joints of many of my other applications, and I just don’t know it.