First of all, it should be pointed out that Gallup is currently showing a 7.5 percent unemployment rate, which is even lower than BLS.
Unemployment rates (unlike employment levels per the establishment survey) aren't adjusted, so the lower volatility of the BLS estimate can't be explained by post facto smoothing. More to the point, relatively more stable measures would have more credibility for such a broad measurement of millions of people, that you wouldn't expect to surge and then plunge the way the Gallup data sometimes does.
The household survey is actually the Current Population Survey, a monthly survey of 60 K, as you noted. It isn't only used to report unemployment numbers, it's used for all labor force statistics, has been around since 1940, and is commonly used in social science since it is panel data.
This means that the BLS identifies about 72,000 households, makes repeated attempts to contact them by physically going to their house, and finally reach about 60,000 households, achieving a response rate of between 92 to 93 percent. Many of the surveys are actually conducted in the respondent's house in person by professional interviewers, and others are conducted over the computer.
They then go back and interview the same people for 8 months, yielding better continuity of data and estimates of change. Gallup doesn't report its response rate, but it does say they rely on random digit dialing of listed telephone numbers, which means their response rate is almost certainly very low as is the case for all telephone polls. CPS is undoubtedly one of the most accurate surveys in America today, Gallup can't compare.
In fact, I found it baffling that Gallup proposed replacing CPS with its P2P measure (for one, they measure different things, and Gallup never explained why their measure is better, besides saying that it's "simpler") because in their own methodology description, they write, Demographic weighting targets are based on the March 2011 Current Population Survey figures for the aged 18 and older-- so if CPS were done away with, they would not be able to weight their own P2P.
There's a piece of information in there that's of interest. The BLS surveys the same households for 8 months in a row. Are there 60K new each month for a total of 480K in the pool? That's not how I read it. Otherwise they have 60 K total with roughly 8K rolling off each month. If so then that certainly accounts for the unusually smooth curve they have given the data.
However, it belies the level of systematic error that is inherent in that type of sampling. An initially skewed sample propagates through the pool but isn't reflected in statistical variations. A proper display of the data would separately cite the statistical and systematic errors separately so that the reader would know to read greater error into the measurement than appears due to the intrinsic fluctuations.
Generally this is not a preferred technique precisely because it hides one of the sources of error in the data. A straight random sample more clearly exposes the natural fluctuations of the sample. I recognize that lots of pollsters like a rolling sample precisely because it smooths the data and is cheaper in terms of contact hours. In terms of measurement theory a single larger statistically independent sample is preferable. The reader can inspect the raw fluctuations to test if they make statistical sense, and if a smoother curve is desired for display it can be constructed from the statistically independent data points.
