Science, Stories, Etc.

7 Gpeople

2011-08-20T12:11:00.000-07:00

While at a conference in Istanbul, I went to the İSTANBUL ARKEOLOJİ MÜZELERİ, which was an absolutely fascinating archeology museum. Istanbul has featured prominently in the growth of civilization, and I struggled to keep track of the many different civilizations and cultures that occupied the region at one time or another. I had to find a youtube video to help me sort it out.

There was a very nice exhibit on Troia (Troy), which apparently really existed; it's ruins were unearthed in a farm field not too far from here. It was settled, destroyed, and resettled in 9 different epochs before being ultimately abandoned. Even back then, it seems that anything you dug up had a 1000-year history. Buildings were built on top of buildings. The reconstruction of the different settlements was interesting.

It was really hard for me to get my head around how small cities were in comparison to now. Istanbul currently has 13 Mpeople living in its greater metropolitan area. In 3000 BC, that was the human population of the world. The largest cities in antiquity were ~250 kpeople. It made me wonder if all of our advancements in technology and culture in the last couple hundred years could be attributed strictly to 1) more people to do the work and 2) longer tails on the normal distribution of people with various abilities.

So just how many people were there as a function of time? The log-log plot above from wikipedia shows current best estimates. Apparently, some 70 kyears ago, possibly as a result of a major volcano eruption, the human population was reduced to something on the order of 1000 to 10,000 "breeding pairs." Since then, the population rapidly recovered to several million, where it remained stable until agriculture was developed. This is all in a nice video tracing genetic migration via mDNA.

Since then, there has been exponential growth (a line on log-log plots) with a transition to a slower growth coefficient at ~400 BC. Occasional Black Plagues aside, the human population has increased dramatically. I remember hearing once that half the people who ever lived are alive right now. That's actually definitely false--it's closer to 6%. Also, everyone seems to think that population growth is accelerating (remember this video from the 80's ?). The graph above definitely shows that's not true, either.

But what is true is that this growth cannot continue unchecked without hitting its head on something, be it food supply, global warming, danger of pandemics, warfare, declining birth rates, or whathaveyou. It's estimated that in October of this year, 2011, there will be 7 Gpeople on the planet. This map shows where the population currently is, but it's estimated that much of the growth in the next century will happen in poverty-stricken Africa. Things pretty much have to plateau around 10 Gpeople, though.

So what to do? The most effective ways to reduce birth rates, which is key to controlling population growth, global warming, saving the environment, and many of the rest of our problems are:

contraception
improving the standard of living (ending poverty)
education (and education about contraception)
and reducing infant/child mortality

That last item is counter-intuitive. The reason it is important is that when survival rates are low, couples have more children to compensate, including a buffer for uncertainty. Having a predictable path from birth to adulthood allows for more precision in family planning.

The "New" Zodiac and the Earth's Spin

2011-01-14T10:44:00.000-08:00

The latest rage in popular astronomy seems to be the realization that the Sun may now move through 13 constellations instead of 12. I haven't personally checked this, so I'm going to take their word for it. What's bothering me is something else that's being repeated in connection--that this is "due to shifts in the earth's rotation and orbit", or more flagrantly, that "since the zodiac periods were established millennia ago, the moon's gravitational pull has made the Earth 'wobble' around its axis in a process called precession".
Someone (I guess, me) needs to make it clear that any shift in the apparent path of the Sun through the background constellations can only be caused by a change in the inclination of the Earth's orbit (possibly as a result of the precession of the Earth's slightly inclined orbit around the Sun), not the precession of the Earth's axis of spin.

The reason the Sun appears to move through background constellations is because we're moving around the Sun (see figure to left). From the perspective of Earth (blue dots), the Sun (orange) appears in front of a different set of stars at different times of year, when the Earth is at different positions in its orbit. The project of the Sun from the perspective of the Earth is shown with dashed lines.

Now the thing is, that dashed line from the Earth through the Sun doesn't depend at all on the Earth's spin. To see this, imagine you remove the Earth from one of those locations and instead place yourself floating in space. You could turn any which way you want--upside-down, rightside-up, twisted alley-oop--the Sun is going to be in the same place relative to the stars behind. Similarly, the Earth's spin (and any precession in that spin axis) can change the orientation of the Earth, but makes no difference in what stars the Sun appears in front of.

What actually can change where the Sun appears relative to background stars is the physical location of the Earth.
This is a change in the Earth's orbit, not spin. One way to change where the Sun appears relative to background constellations is to move the Earth to different places in its elliptical orbit. This will move the Sun through the standard set of zodiacal constellations. In order to move the Sun out of the normal zodiacal progression, you need to move the Earth up or down. The way this can happen is if the plane of the Earth's orbit precesses around another axis, so that the "high" point in the orbit moves slowly around the Sun.

So to clarify, if the Sun appears to move through a new constellation, it is because the Earth's orbit around the Sun has changed, not because the Earth's spin has changed.

Distribution of Actual Births Around Estimated Due Date

2010-12-28T14:00:00.000-08:00

Given that we're now in overtime here for our second child, I have become very interested in the question of what our "due date" meant in the first place.

The plotted on the left are data from a study of Canadian births, 1972-1986 (Arbuckle & Sherman 1989). I then started trying to aggregate some data for my own plots.

The first thing I noticed required a little attention was the "fence post" problem relating to recording gestational age. If a study A records births in the 39th week, where exactly is that on the x axis? Well, if we assume that they started counting with 1, then the 39th week is actually 38.5 +/- 0.5 weeks (they are counting fence). But if study B records births from week 39-40, the implication is that they started at 0 (they are counting fence posts), so 39-40 is 39.5 +/- 0.5. That was a little tricky.

Then we have studies that bin over different time intervals. If another study C records births at 37-41 weeks, how do we relate that to A and B? The intelligent way to plot this would be to use the probability density of going into labor, which divides out by the length of time over which the observation was made. So we should be careful to do that.

And then there's just the general problem that a lot of studies list percentages for births in each time bin, but don't list total populations or error bars, so we don't know what the errors in their measurements were. Ugh. So I crossed my fingers and hoped they followed good practices with their significant figures: I assigned an error equal to the last significant digit they list.

I got most of my data from this semi-thorough compilation of census data and going to some of the original sources. The data aren't great, but they are adequate (see plot to left). I fit to the data an increasing exponential tail to the left, plus a normal distribution. The fit isn't great (despite some claims in the literature of it being normally distributed), but it captures enough of the overall distribution.

The width of the normal distribution was 1.5 weeks, centered at 39.5 weeks. This seems consistent with several sources that suggest ~10% of pregnancies would go into the 42nd week if they were allowed to. It also suggests that the "due date" means the mean of the normal portion of the distribution. Yet fully 1/2 of pregnancies will go beyond the expected due date, and 1/6th will go past 41 weeks, according to this coarse fit.

Of course, none of this accounts for biases that we know exist. The growing prevalence of inductions and C-sections move births earlier artificially. Although the statistical significance may questionable owing to systematic biases (self-selection for uncomplicated pregnancies, etc.), it appears that the recorded midwife births go later than the aggregate (presumably hospital-dominated) births at the 2-sigma level. This may potentially indicate that without intervention biases, the distribution of birth dates around the expected due date could be broader and weighted toward later dates.

Listening to the Astro2010 Decadal Survey

2010-08-13T08:18:00.000-07:00

Here are some streaming comments as I'm listening to results of the 2010 Decadal Survey. A shortlink to the report is here.

11:10 am EDT: Excellent placement of "what were the first luminous objects, and when did they form" second from the top of "Major questions to address this decade".

11:20 am EDT: Ooh, even better. "Cosmic Dawn" is the first of the 3 over-arching fields. Good fielding for EoR as a top science priority in the next decade!

11:23 am EDT: Now we're starting into descriptions of the panels. I was involved in several of the RMS (Radio, Millimeter, and Submillimeter) submissions, and am looking for HERA (the Hydrogen Epoch of Reionization Array). I'm also rooting for the Allen Telescope Array's Radio Sky Surveys Project. Finally, I'm hoping for some mention in the TEC (Technology development) program of prioritizing the development of shared solutions to digital signal processing hardware and libraries, which was the recommendation of a white paper I drafted with the help of the CASPER community.

11:45 am EDT: Mention of SKA as a priority for Radio Astronomy, unsurprisingly.

11:47 am EDT: Looks like Roger's wrapping up here. Turning to Q&A.

Meanwhile, I'm reading through the report. I see radio instrumentation is listed as a funding priority on ES-4, linking to 7-39. A good sign.

A painful line on 1-18: "U.S. participation in projects such as the Square Kilometer Array is possible only if there is either a significant increase in NSF-AST funding or continuing closure of additional unique and highly productive facilities." Ouch.

But on 2-12, my own sky map (well, with "permissions pending" for now). Now that's something!

12:01 pm EDT: An interesting question. "Why such a priority on habitable planets in the decadal review?" Sounds like the answer is that it's particularly primed to make big breakthroughs. I think I agree with that. I wasn't surprised that it was on there. There's some rumors going around that Kepler has found earth-like planets in earth-like orbits.

12:02 pm EDT: What about the surplus of post-docs relative to faculty positions? Answer: there are a wide range of careers and positions available to astronomers, so it's not unreasonable to have a larger number of post-doc positions where budding astronomers get training. I'm not sure that answer fully appreciates the scale of the problem.

Continuing with reading, on page 3-13, "The HERA program, a project that was highly ranked by the RMS-PPP and included by the committee in its list of compelling cases for a competed mid-scale program at NSF, provides a development pathway for the SKA-low facility. Progress on development of the SKA-mid pathfinder instruments, the Allen Telescope Array in the U.S., the MeerKAT in South Africa and the ASKAP in Australia, and in new instruments and new observing modes on the existing facilities ... will provide crucial insight into the optimal path towards a full SKA-mid." That's good to see mentioned. Sounds like it lays the groundwork for a strong future proposal to get funded. It's not a promise of funding, though. Not that such a promise was expected.

12:11 pm EDT: Second use of "tripwires" for projects. A very colorful phrase.

Interesting plot on 4-15: papers in all astronomy fields are increasing. Instrumentation papers seem to be low in number, but holding their own against other fields (same percentage contribution to total paper number).

On 5-14 for Data Reduction and Analysis Software: "Flexibility, openness, and platform independence, modularity, and public dissemination are essential to this effort. Focused investment in a series of small-scale initiatives for common tool development ... may be the most cost-effective approach, although there are undoubtedly synergies with the pipeline development needed for the large-scale projects." Sounds helpful for some of my projects like AIPY, SPEAD, and CASPER.

12:18 pm EDT: Comments on the SKA? Answer: SKA is the future, but the US can't pay for the construction on the proposed time scale (but slower might be ok). Technology development should be prioritized though. Low-frequency SKA, though, targets EoR, and we're interested in projects targeting that. Yes!

On 5-21 for Technology Development: "The committee received community input in the form of white papers on the funding needs for technology development in areas such as ... high speed, large N correlators. In these areas and others, researchers ... had come together to plan a coherent strategy for the decade. The OIR and RMS panels made a convincing case that the current level of ATI funding needs to be augmented in order to successfully pursue these highly-ranked technology development programs and roadmaps." Looks like my white paper fell on receptive ears.

12:28 pm EDT: Neil Tyson is closing down Q&A. He is one cool dude. I'm glad he was on the panel.

On 7-7 for Science Objectives for the Decade: "Find and explore the epoch of reionization using hydrogen line observations starting with the HERA telescopes that are already under construction." Wham. And in Table 7.1 on 7-32, Priority 2, Projects thought compelling: HERA.

On B-2 for Program Priorization, in Table B.1, I see both ATA and HERA. I'd say ATA didn't necessarily win big in the review, but at least they're there.

And finally, in appendix D-1: "Hydrogen Epoch of Reionization Array ... is a multi- stage project in radio astronomy to understand how hydrogen is ionized after the first stars start to shine. The first phase (HERA I) is under way and will demonstrate the feasibility of the technical approach. The second phase (HERA II) would serve as a pathfinder for an eventual world-wide effort in the following decade to construct a facility with a total collecting area of a square kilometer and the power to make detailed maps of this critical epoch in the history of the universe. Proceeding with HERA II should be subject to HERA I meeting stringent performance requirements in its ability to achieve system calibration and the removal of cosmic foreground emission." We've got our work cut out for us!

Don Backer

2010-07-27T13:50:00.000-07:00

Sadly, the world and I lost Don Backer on July 25, 2010. In addition to being an inspiring scientist, instrumentalist, and educator, Don was my graduate and post-doctoral advisor, and my close friend. He and I worked closely together from 2004 until last Sunday, when he died suddenly of an apparent heart attack.

Don was well known for discovering the first millisecond pulsar early in his career. More recently, he and I had been working on the Precision Array for Probing the Epoch of Reionization--and experiment for detecting the first stars and galaxies that formed in the universe. We recently had a made a lot of exciting progress with this experiment, and it is especially tragic to lose him at such a pivotal time.

Don was a very warm yet reserved man. He was always extremely busy, and I envied his ability to juggle a huge number of tasks at once. Yet every time I walked into his office, gave me a big welcoming smile, saying "Hi Aaron! Come on in." In that instant between when he looked up and when recognized me, I would sometimes see a hint of displeasure at being interrupted (a lot of people in the department walked into Don's office to hassle him about any of the many projects that he was involved in), but it was always gone the instant he recognized me, and I took pride in being someone from whom Don welcomed interruptions.

Don was always a model to me of how to be and instrumentalist and a scientist. I've long been interested in both building and using scientific instruments, and Don was a shining example of how to do both. I learned a lot from Don that helped guide me professionally, and I owe him a lot for his advice and generosity. It is sobering to consider that my next steps will have to be without Don's quiet support and encouragement. In many respects, though, Don's generosity has already helped pave the next steps for me. I'm sure I will continue to incur debt to him for years to come.

One of my favorite qualities of Don was his grand sense of adventure. Our foray into the Karoo desert to deploy PAPER in South Africa could not have happened without Don's enthusiasm for traveling, roughing it, and flying by the seat of the pants. I loved going on deployment expeditions with Don. He was always bright-eyed and smiling, summoning such energy at 66 years that I, at 29, struggled to keep up. It was not hard to see the Don of the black-and-white photographs, the same wiry energy and wry grin that stood in front of me.

Don was never very forthcoming with advice--he advised me more by example. I'm pretty sure this was a result of a very ingrained sense of humility. Don never said "you're wrong", or "you should". I think he didn't feel it was his place to pass judgment on people. Despite this humility, or probably because of it, Don was an effective leader. Without badgering people or using heavy-handed methods, Don brought people into consensus and helped move projects forward. Unfortunately, his effectiveness, coupled with his self-described "responsibility gene", meant that he was often called upon to bail out troubled projects, and he had a hard time refusing them. I often wished Don spent more time on PAPER. I think he did, too.

Don and I were a great team. I'm not a good multi-tasker. Don insulated me from a lot of project management, logistics, and distractions, carving out a space for me to work effectively toward our goal. Soon, some of the important products of our partnership will bear fruit, and I'm sad that Don won't be there to see it. But he knew it was in the works before he left, and for that I am thankful.

I'm sad to have lost a good friend and mentor. Things are hard now as we try to pick up the pieces of all the many things Don was managing. I'm sad that he's not here to help. He was always good at bailing us out.

Who Dominates Health Care Costs?

2010-03-29T09:53:00.000-07:00

There's nothing like a little bout with MRSA to make one pay a little more attention to the state of health care legislation. Two nights in the ER are definitely making me thankful for health insurance. Knowing that it wasn't going to cost me an arm and a leg to get antibiotics through an IV (in fact, it probably saved me the leg) definitely helped me to seek care early, rather than waiting for the infection to get truly life-threatening. And that probably saved in health care costs in the long run.

I've heard it argued many times by the other side that universal health care will drive up the cost of health care for everyone, because so-called "healthy people" will be paying, through their premiums, for the bills of the "unhealthy". Ignoring that:

the above is a tautological statement about what insurance is
people routinely go from the "healthy" group to the "unhealthy" group and back again
we should maybe feel a moral obligation to care for the unhealthy

Yeah, ignoring that, I wanted to know if the underlying assumption was, in fact, true. Who dominates health care costs? Is it the small number of extremely sick people? Or is it the larger number of moderately sick people?

To answer that question, I went searching for the population distribution of health care costs. I found the following publication: Variations in Lifetime Healthcare Costs across a Population (Forget et al. 2008). To the left are reproduced Figs. 4 and 5.

Given all the hype, I was somewhat underwhelmed to see that these curves depict (with the exception of an excess at the lowest cost bin) a gamma distribution. This isn't surprising, because a gamma distribution is supposed to represent the sum of a bunch of exponentially-distributed random variables.

To find the contribution of people in each cost bin to the total health care cost of the population, we simply need to multiply the population of that bin (drawn from a gamma function) by the mean health care cost of that bin (a linearly increasing function). Setting the mode of the gamma distributon to $90k for females, and tweaking the k and theta parameters (I'll chi-by-eye it at k=4.5, theta=1.0) we get the following distributions of fractional population (black) and fractional total health care cost (red), as a function of lifetime healthcare cost:

So who dominates health care costs? Those just slightly above the mode, which is to say, the large number of people who are just a little sicker than most. And that really could be any of us, folks.

AstroBaki on MediaWiki

2010-02-10T09:15:00.000-08:00

I just started up a new wiki called AstroBaki. The main reason I did this was that my MoinMoin AIPY wiki was clunky to use and was getting spammed lots. I switched to the MediaWiki engine, which has better automated control over these kinds of things. As an added bonus, MediaWiki has support for latex math. This got me thinking...

When I started grad school, I had a hard time transitioning from feeling like I was producing and contributing (I was working as a development engineer for SETI) to just absorbing knowledge. To make myself feel better and more invested in learning, I started doing something for which I became moderately famous around the department: latexing lecture notes on-the-fly. For full disclosure, I should mention that I copycatted the idea of latexing on-the-fly from my friend Phil.

The key to success is to use lots of "defs", and to recognize when you need to def a sequence of commands. When the same sequence of symbols started popping up, I would pretend that I had already def'd the command and start using it, and when there was a pause in the derivation, I would remember to scribble down what that command should mean. In my later years, I also started drawing figures in paint for inclusion in latex.

Anyway, I now have about 4 or 5 latex'd class notes that I have put on my website. From what I hear, they are still regularly used in UCB classes, and I occasionally get happy emails from grad students thanking me for the effort. Meanwhile, I've been reading a book about Nicolas Bourbaki, a famous pseudonym for a group of (mostly French) mathematicians who collaboratively re-wrote mathematics from 1935 to the 70s. Nicolas Bourbaki was a wiki, ahead of its time.

"Now wouldn't it be cool," I thought to myself, "if students using these lecture notes could fix them when they are wrong (after all, they were written on-the-fly), and re-organize them to make more sense?" Could these notes become a sort of open-source textbook for astronomy? So AstroBaki was born.

The difficulty, I am finding, is in translating latex (especially latex heavy in defs) into mediawiki. The best tool I've found so far has been pandoc, which didn't do the defs, but did everything else pretty well. I'm loath to do things by hand, so I'll see what can be automated, and I'll keep you posted.

Where is GCC for FPGAs?

2010-01-22T13:19:00.000-08:00

A lot of the digital signal processing that gets done in radio astronomy these days is done on Field Programmable Gate Arrays (FPGAs), and one of the projects I've been working on from the beginning in my research is developing open-source libraries for programming these chips. My part in this has generally been on the algorithmic/mathematical side: writing FFTs, filters, cross-correlation engines, etc. Another key aspect of this work, though, is a toolflow that allows people to design systems at a high level with parameterized algorithmic cores, and to turn that design into the wiring instructions that tell the FPGA how to implement the system.

We currently use a design entry system based on Simulink running on Matlab, and while it is an extremely powerful environment, we've also found it to be limiting, frustrating, and hard to maintain designs in. In October, I volunteered at an international workshop on astronomy signal processing to explore alternatives to this environment. My current favorite is MyHDL, which uses Python to generate lower-level code in Verilog or VHDL, and I may start looking more deeply into porting a design to use MyHDL.

Something that is bothering me, though, is that however much we work on porting our toolflow open-source equivalents, there is currently no open-source compiler for FPGAs. The state of affairs in FPGA-land is something like PCs in the '70s, when every personal computer had its own specialized compiler. For PCs, the problem was solved by GCC (the Gnu Compiler Collection), which became the default open-source solution for compiling most languages to target the many CPU architectures that exist in the world today.

I'm keeping my eye on gEDA, and notably Icarus, which seems to be a free synthesis tool (synthesis, mapping, and routing are the 3 main stages of compiling for an FPGA). Perhaps mapping and routing can never be open-source, since they tend to be very chip-specific. But here's hoping...

Hands-On Cosmology Education

2010-01-08T09:33:00.000-08:00

Yesterday I spent the morning giving a gosh-wow talk about cosmology to a physics class at Athenian High School taught by my housemate Dave Otten. It was a lot of fun, and the students were all very enthusiastic. It was almost entirely driven by their questions, and they loved being pitched curveballs (time is reference-frame dependent, the universe is expanding, spiral arms are standing waves, etc). The hour-and-a-half lecture was over before we knew it.

Afterward, Dave mentioned that it would be really cool if there were a way to talk about galactic-scale astronomy and cosmology that was in keeping with the philosophy of their school, which emphasizes lab-based, hands-on learning. He mentioned that PhET is a free resource he uses for providing interactive simulations that make hands-on labs out of subjects that otherwise would be too slow, small, big, fast, or dangerous to perform live in a classroom. He also lamented that there aren't any galactic- or cosmological-scale simulators there that could help to understand how systems on this scale behave, and that could perhaps illustrate exactly where the problems of dark matter and dark energy are encountered. Has anyone seen something like this?

The Need for SPEAD

2009-11-09T09:50:00.000-08:00

I've been absent for a good while now as a result of participating in a (successful) deployment of our PAPER experiment in South Africa. The Karoo desert in SA, where we were stationed, was very reminiscent of Rangely, CO where I grew up, except for the occasional baboon or kudu in the road. Though it came at a price of a lot of work piled up for me when I got back, and an awfully long time away from J, the isolation from all but our experiment helped ferment some ideas I'd been having about migrating the AIPY toolkit I've been developing to use a streaming data format that would avoid unnecessary disk accesses, would allow AIPY to be integrated directly with the correlators developed by our CASPER project, and would help our experiment develop a real-time analysis pipeline for compensating for ionospheric distortion in our data.

After chatting with a lot of guys working on the Karoo Array Telescope in Cape Town, we came up with a concrete protocol build on something already being used for CASPER correlator output. I just got done writing my first grant proposal to the NSF, funding a graduate student to work on this protocol--the Streaming Protocol for Exchanging Astronomical Data (SPEAD, pronounced "speed"). The process of writing a grant myself was a learning process, and helped me understand where a lot of the questions I got asked by my previous advisors were coming from.

A lesson I got to take away from SA was this: the reason we were in SA (as opposed to Australia) for PAPER was because we had been working with the KAT team, sharing correlator development. The reason we were working with the KAT team was because CASPER and KAT started up a collaboration a few years before. And that collaboration was started up because Dan Werthimer went down to visit SA some years ago to help advise them in a review of the design of their telescope electronics. Dan was invited there because he struck up a fast friendship with Alan Langman (the KAT director) at an earlier conference. The moral of this chain of causes and effects being that sometimes large projects go in new directions because of personal friendships, and sometimes those friendships end up making the difference in the success of a project.

"Guess What I Was Thinking" Logic Puzzles (A Rant)

2009-09-06T11:37:00.000-07:00

Probably more than most, I like logic puzzles. They're fun. But there's a variety of logic puzzles, especially prevalent in IQ/Mensa tests, that I really dislike. They are the "what's the next symbol in the pattern"-type of puzzles, and I HATE them. They are written like there is one (and only one) answer, and you must be dense not to see it. But can't I put anything in that blank space and call it a "pattern"? And what if the pattern that I see when looking at the provided sequence isn't the one you were thinking of? Is anyone with me in recognizing that these aren't logic puzzles at all? They're "guess what I was thinking of" puzzles! They aren't adequately constrained. They don't specify the parameter space from which the sequence is drawn. Anything can be a pattern! Anything!

What they actually want you to do is find the most likely symbol given several measurements and a set of priors about the likelihood of the author picking a particular sequence, but they neglect to provide you with any information about those priors. Maybe they assume that you can guess the priors based on estimates of your own sequence-picking priors, but that only works if your brain works the same way as the authors'. And quite frankly, if the authors can't appreciate that answers to these puzzles they are writing are indeterminate, I'm pretty sure their brain isn't working the same way as mine. Quit calling these logic puzzles! Put them on a Berkeley Psychic Institute entrance exam, not a college entrance exam.

Ok. Done ranting.

Graph-SLAM

2009-08-26T10:16:00.000-07:00

After a trying, but ultimately successful month spent extracting the family from Puerto Rico and re-embedding us in Berkeley, I'm just starting to get back on top of things enough to think about posting...

I had lunch yesterday with a good friend of mine, Pierre, who co-founded a company that specializes in sensory and mapping systems such as those that are used to create Google's "Street View". I was impressed to learn about their system for combining data from GPS, LIDAR, car odometers, and IMUs to create a consistent picture of how a vehicle is located and oriented in space as a function of time. They've spent a lot of time calibrating their systems, and use some sophisticated MCMC post-processing methods for deriving the actual trajectory of a vehicle.

Although the antennas in the PAPER array (that's the low-frequency interferometer I'm working on), are much less mobile than a car, there was considerable overlap between the problem Pierre has been working to solve and the calibration problem I am facing the requires positioning antennas and celesital sources as a function of time in the face of ionospheric distortion, variable gains, etc. Pierre pointed me to Graph-SLAM as a formal description of the problem that we are trying to solve, and suggested that Kalman Filtering with RTS Smoothing was a powerful technique for converging to the optimal solution (with covariance information) in linear time.

The Voynich Manuscript: Bootstrapping Language

2009-07-14T17:37:00.000-07:00

The internet is a powerful and dangerous thing. It all started when I read the latest xkcd, which warned me that visiting cracked.com was dangerous. Then I read about the "6 phenomena that science can't explain" (which was a very dramatic title for some underwhelming mysteries), and next thing I knew, I was reading about the Voynich Manuscript. I learned about cryptography, glossolalia, the Manchu language, among other things. Then I took a look at the manuscript and before I knew it, I had a transcribed version of the manuscript in electronic form using the European Voynich Alphabet. And it just went downhill from there.

To summarize, the Voynich Manuscript (hereafter VMS) is a handwritten text with some illustrations some 500 years old. It uses glyphs no one knows how to read, it is not clear if it corresponds to a known language, it may or may not be encrypted, and little progress has been made in deciphering any of it, despite the fact that some bright people have tried. So I decided to have a crack at it.

The reason I got interested is because of the similaries to SETI. Arecibo, back in 1974, transmitted a message off into space that had been designed to be decrypted. We might receive a message like that some day. Or we might intercept something much like the VMS--a bunch of data in a language that we have no prior knowledge of--and we may be finding ourself trying to figure out how to bootstrap a language. That is to say, to learn the grammar and semantics of a language from a static example, without outside help.

Is this possible? For grammar, I'm pretty sure of it. I can imagine an algorithm (maybe Maximum Entropy Modeling and Bayesian learning applied to grouping and parsing) that uses correlations in the appearances of language elements (starting with letters and building up) and correlations in the behaviors of these elements relative to one another to build a model for parsing a language. For the VMS, I used something similar to this (not the MEM and Bayesian part) to show that spaces, line breaks, and paragraph breaks show similar grouping correlations relative to other VMS letters, and so can probably be considered one grammatical element of whitespace. That's a pretty simple thing to deduce, but it was actually something I was worried about in getting started with the VMS.

Sematics is another issue. Once upon a time, I would have had an optomistic answer to bootstrapping sematics from text that was not written for that purpose. However, after watching my child mysteriously acquire language, illustrating how hard-wired the human brain is for learning language from another human, and how much it relies on shared experience and feedback, I'm less sure.

I would be interested to know if there is a field of mathematics that studies sematics and the properties that a self-contained system needs to have to be able to generate sematical relationships. The Arecibo message relied on a shared physical environment to try to bootstrap sematics. I wonder if it would be enough to describe the rules of the grammar of a language in the language itself. That one, once the reader had deduced the relationships between elements, you would have a shared knowledge of that subject that might enable a reader to correlate the structure of the descriptions with the grammatical structure and thereby establish the first sematical relationships.

Anyway, after preliminary analysis of the VMS, I'm pretty sure that it's not random gibberish (there are correlations between elements on levels ranging from letters to words), and if it's encrypted, it's a weak form of encryption that preserves these correlations. My pet theory, extended from the glossalalia idea, is that this is actually plaintext in a natural language with an invented set of symbols, but that the natural language might be the accidental or intentional creation of a savant or scholar.

Countability and Strong Positive Anymore

2009-07-09T06:24:00.000-07:00

Seven or eight years ago, I discovered that I have a linguistic condition called "Positive Anymore." I was in college, chatting with my roommates, and said something like "Anymore, I just lift weights in the Leverett gym." One of my roommates just couldn't take it anymore. "I've heard you say 'anymore' like that for years now. What the hell does it mean?" A quick poll of those present revealed that I was the only one for whom that construction made grammatical sense. My aunt (a linguistics professor) gave me the prognosis: I had Positive Anymore.

I used to think that PA was a linguistic shortcoming of mine, but anymore I'm convinced it's more like a superpower. Whereas most English speakers can only use the word in negative constructions like "I don't drive anymore" I have the uncanny ability to use it positively as per the first sentence in this paragraph. Moreover, I don't just have PA, I have strong PA, which means I can, at will, detach "anymore" and put it anywhere in a sentence. "Anymore, I just take the bus." Astounded yet?
If you're still having trouble parsing that, replace "anymore" with "nowadays"--to me, they mean about the same thing.

I receive no end of flak from friends, relatives, and spouses about my PA, although it's not that uncommon a condition. In fact, I have caught several of my relatives (mostly on my dad's side) using PA even after making fun of me for my PA. They have it and don't even know it.

Anyway, S and I were trying to figure out if my use of PA was inconsistent with my use of "any", which I use according to the standard rules. But it turns out the standard rules are weirder than you might think. For example, "I don't want any spam" is a grammatical negative construction; "I want any spam" is ungrammatical and positive. "Do you want any spam?" is grammatical and seems positive, but it turns out that there is an implied negative in English owing to the uncertainty inherent in questions and subjunctives. Hmph. And then what about "I like any spam I can get?" That seems positive again, but the clause "I can get" is required to make it grammatical. And then the plot thickens. "I feed spam to any dog" is grammatical and did not require a clause to modify "any dog."

Our theory was that using "any" positively without a clause requires the noun modified by "any" to come in quantized units--to be countable. Dogs are countable; spam is a continuum, much like water or space-time. The best example we could think of that illustrated this was "fish". Fish can be countable noun (number of live fishes) or continuum noun (amount of dead fish to eat). If I say "I'll take any fish," the ambiguity in the countability of fish is broken--it's clear I'm talking about a live fish (or, perhaps, a type of fish, which is also countable). But if I say "I'll take any fish that you give me," the ambiguity is preserved.

The implication was that for my use of PA to be consistent with the standard use of "any" (which it may not need to be, since "anymore" is one word, not "any more"), I must be thinking of the time interval referred to by PA (i.e. now and continuing indefinitely into the future) as something countable rather than continuous. Maybe. I don't know. Anymore, I'm just really confused.

The Need for Speed

2009-07-02T07:56:00.000-07:00

On the drive from San Juan to Arecibo this morning, I got to wondering about where my average driving speed fell in the distribution of drivers here in Puerto Rico. In the states, I felt like I was a pretty average driver, but here en la isla, the distribution of driving speeds is different. There are a lot of fast drivers, too be sure, but there is also a subpopulation of drivers whose speed is significantly (~10 mph) below the speed limit. This may be because relative to the US, PR is economically depressed and so more old cars are on the road, or as a reaction to the more erratic driving habits there seem to be here, but anyway, I definitely pass more people than pass me now.

So in an effort to discover where my driving speed fell relative to others (and in and effort to alleviate the boredom of driving 1.5 hrs alone), I started counting how many cars I passed and how many passed me as I was going 65 mph (the speed limit). Out of 55 pass events, only 9 involved me getting passed. To make this a tractable problem in my head, I decided to assume that driving speeds were normally (gaussian) distributed about a mean--even though this contradicts my anecdotal evidence above. Using this approximation, my first instinct was to say that 1/6 of the cars on the road were faster than me, and since ~2/3 of samples are within +/- 1 sigma of the mean in a gaussian distribution, 1/6 of the samples would be above +1 sigma. So I approximated that I was a 1 sigma driver.

But then it occurred to me that I needed to control for a significant sample bias. This is because the test I was doing wasn't randomly selecting cars and comparing my speed to them. Cars were far more likely to get selected if the difference between their speed and my speed was large. A car going the same speed as me would never pass me, and I would never pass it. But I would assuredly pass almost every car on the road that was going 10 mph as I went 65. The "road distance" that I sampled for different velocities is proportional to abs(v-v0), where v0 is my velocity. The effect this had on my samples was to underweight speeds close to my own and overweight the wings of the gaussian distribution I had assumed as my model. If I drove exactly the mean velocity, this effect would not be terribly important--if the model were correct, it would still be the case that as many cars passed me as I passed. But as my velocity moves away from the mean velocity, the "normal drivers" who are only going a little faster than me get undersampled, so I only see the drivers who are tearing around like a bat out of hell. At the lower end, I still see the real slow-pokes on the road, but I start seeing people who are going a bit faster than that, of which there are a lot more. The effect of this sample bias, it seems, would be to make it seem that I'm a farther outlier in my driving speed than I actually am.

So now, to figure out where I fall in the (normal) distribution of driving speeds, I need to know exactly what the mean driving speed and what sigma is, so that I can compensate for the abs(v-v0)
sampling factor. That means I need to figure out 2 numbers, but unfortunately, I only measured 1 number (that 1/6 of the passes while driving were me being passed) so I won't be able to properly constrain this problem. However, I should be able to figure out the mean on my drive home by finding the speed at which as many people pass me as I pass. For now, let's say this is 55 mph. Then all I need to do is find the sigma for which a gaussian distribution around 55 mph downweighted by abs(v-65 mph) has 1/6 of the area lying above 65 mph. I just solved that numerically on my computer, and it's saying that the best-fit sigma is ~22 mph. So that puts me at about +1/2 sigma. That seems reasonable.

An interesting next step (which I'm not going to do right now since I need to get to work) would be to translate the sample error in my pass measurements into an error in the determination of sigma, and then the error in my driving speed percentile.

What are the chances?

2009-06-02T06:24:00.000-07:00

I cringe every time that I hear this phrase. I heard it most recently when my friend had her car stolen from San Juan. It was recovered in a semi-drivable state in Bayamon. She invested a couple thousand dollars to get rid of the "semi", only to have the car re-stolen a couple of months later. This time, when the car was recovered in Dorado, there was no "semi" to be had, so she's currently trying to sell it for pieces. While the police were fairly understanding (though less than helpful) the first time her car got stolen, the second time was occasion for all sorts of raised eyebrows and skepticism. And in exasperation my friend uttered the phrase in question.

"What are the chances" is a Pandora's box of bad statistics. Statistics is about hedging your bets given incomplete information, but this phrase is always uttered after the fact, when we have (relatively) complete information: it happened. So unless you plan on repeating the experiment, the chances are one. It happened.

As an example, let's take the famous Goat/Car Puzzle. There are 3 doors; one has a car behind it and the other two have goats. After you pick a door, the game-show host opens one of the other doors and reveals a goat. You are then offered the option of switching your choice to the other door. If you play this game repeatedly, you'll win more often if you switch your choice. But the instance just played out, the car was behind one of the doors, and if that was the door you picked, your chances of getting the car were 1. If you didn't, your chances were 0.

You might object: "What were the chances beforehand, when I didn't know where the goat and car were?". But to do that, you need to make some assumptions. You need to assume that at each playing of the game, the cars and goats are randomly assigned and/or you randomly pick doors. Otherwise, it might be the case that the car is always behind door #1, and you always pick door #2. Your chances of success wouldn't be so good in this case. You might have decent prior knowledge of how cars, goats, and doors are picked in this example, but for everyday occurrences, we usually have much more limited prior knowledge. Are cars randomly stolen, or are certain brands targeted? Are certain areas targeted? People often assume that these processes are random, but they rarely are. With limited priors on these events, the question "what are the chances" can't be answered with any certainty and any answers given should be taken with a great big shaker of salt.

Furthermore, people have selective attention. We ignore whole heaps of ordinary outcomes and only pay attention to ones that strike us as interesting. As a friend of mine once said: "Low probability stuff happens pretty regularly because stuff is happening all the time." Even if the processes involved are random, unlikely outcomes are to be expected if the processes are repeated often enough. People tend to ignore the ordinary outcomes, exclaim at the extraordinary ones, and then assume that something deeper is afoot. In my friend's case, the police started wondering if she was being personally targeted or if she was really bad at locking her car. But even if we assume a random model of car thefts, some number unlikely outcomes doesn't automatically imply that our random model is wrong.

Finally, we also need to keep in mind that in complex systems like real life, there may be a huge number of possible outcomes. But something has to happen. When you roll a die, each number only has a 1/6 chance of coming up. Would you roll a die once and then exclaim: "Wow, it came up six! What are the chances?" In real life, there might be millions of outcomes, each with one-in-a-million chance of coming true, but the fact that one of them happens shouldn't be surprising.

Fighting against all of the pitfalls inherent in asking "What are the chances?", I've developed a reflexive response: "What are the chances?" One.

Ida y Vuelta: a Tail?

2009-05-21T04:38:00.000-07:00

It was all over the news and the net yesterday: the missing link has been found! Darwin has finally been proven right!

Hold on a minute. There's no "missing link"--we have example after example of the evolutionary forebearers of Homo sapiens. Evolution was not in question--we already knew Darwin was right. Ida (the name of the fossil found) is not a direct ancestor of humans--the fossil represents a transitional form between lemurs and other apes.

I do not mean to denigrate what is obviously a very important and exceptionally well-preserved fossil that may provide important insights into primate evolution. My objection is to the media frenzy that gives the impression that this is the final resolution to a scientific debate about evolution that a) never existed and b) would not have been resolved by the fossil in question if it did exist.

We have Neanderthals, Homo erectus, Homo habilis, Australopithecus afarensis, and more. Why is Ida, who isn't even our evolutionary forebearer, the "missing link"? I have a theory: it's about the tail. I think that somehow, through all the monkey-human-common-ancestor debate, through the discovery of early hominids, through all of the discussion about increasing cranial capacity and tool usage, everyone has really been wondering "what about the tail? What happened to the tail?" The media was holding out for something it could tout as a human ancestor (or something close enough) with a tail to finally declare the "missing link" found and the evolution debate settled.

You might say that this is all for the best--that the false debate, however belatedly, is finally being put to rest with one last hurrah. But here's a scenario that makes be cringe just to think about it: suppose it turns out that this fossil, which was recovered from a private collection, turns out to be something other than the specimen that the scientist in question thinks it is. Scientists makes mistakes--that's what peer review is for. Then suddenly we've breathed new life into a misconception about evolution that has been hanging around for way too long already.

Golomb Rulers/Squares/Rectangles

2009-05-12T05:39:00.000-07:00

Yesterday I came across an interesting example of the isolation of academic fields from one another.

A common design parameter for antenna arrays is to try to obtain uniform coverage of the aperture plane to get as many independent measurements as possible. Interferometers sample the aperture plane at locations that correspond to the difference vectors between antenna elements. For example, in one dimension, if I put four antennas at locations (0,1,4,6), then that array would sample the difference set of those positions: (1,2,3,4,5,6). However, if I put antennas at (0,1,2,3), the difference set would only include (1,2,3) with 1 occuring 3 times and 2 occuring twice. For the purpose of uniformly sampling an aperture, redundant spacings are lost measurements. We're looking for a minimum-redundancy array.
There have been a few papers in radio astronomy on minimum-redundancy arrays for the more useful 2-dimensional case, including Golay (1971), Klemperer (1974), and Cornwell (1988).

Thinking that this might be a mathematical problem of interest, I ran it by a good friend of mine: Phil Matchett Wood--a mathematician at Princeton. He quickly uncovered the equivalent problem as formulated in math literature: Golomb rulers in 1-D and Golomb rectangles in 2-D. Some relevant papers on the subject are Shearer (1995), Meyer & Jaumard (2005), Robinson (1985), and Robinson (1997). These papers are on the exact problem and describe applications to "radar and sonar signal detection", obviously referring to the need for independent aperture samples in radar and sonar interferometers. Somehow, the differing nomenclature between these fields was never quite bridged, and so there has not been and cross-referencing between these two formulations of the same underlying problem. People like to talk about the possibility that relevant research in one field goes unnoticed by other fields. This is the first time I've across it myself, though.

Compressed Sensing and Wiener Filtering

2009-05-05T06:10:00.000-07:00

Today I'm trying to expand my understanding of how we can best remove contaminant signals from the data we take with the Precision Array for Probing the Epoch of Reionization (PAPER). There is a specific problem I want to make sure we can solve for PAPER. Foregrounds to our signal, particularly synchrotron radiation, are expected to be very smooth with frequency. The idea put forth by the MWA and LOFAR groups is that by observing the same spatial harmonics at multiple frequencies, we should be able to remove such smooth components to suppress them relative to the cosmic reionization signal we are looking for. However, generating overlapping coverage of spatial harmonics as a function of frequency is expensive. My intuition is that since foregrounds do not have a spatial structure that changes dramatically with frequency, we shouldn't need to sample a given spatial harmonic very finely in frequency to get the suppression we want. This would allow us to spread our antennas out a little more and get measurements of the sky at a variety of spatial modes.

In many ways, our problem is analogous to what was done with the Cosmic Microwave Background (CMB). For foreground removal in CMB work, Tegmark and Efstathiou (1996) begin with an assumption that foregrounds can be described as the product of a spatial term and a spectral frequency term. This allows them to construct Wiener filters that use the internal degrees of freedom of their data, together with a model of their foreground and a weighting factor based on the noisiness of their data, to construct a filter for removing that foreground. For the most part, this is standard Wiener filtering, except they have to be careful about what they do to their power spectrum, so they apply a normalization factor to correct for a deficiency in Wiener filters. Tegmark (1998) goes on to generalize this technique for foregrounds that vary slowly with frequency. I'm in the process of wading through these papers, but they seem to be directly applicable to what we are doing, and seem to confirm my suspicions that synchrotron emission should be well-enough behaved to require only sparse frequency coverage of a wavemode in order to be suppressed.

Another tactic that I am investigating is that of compressed sensing which I was alerted to in talks by Scaife and Schwardt at the SKA Imaging Workshop in Socorro this last April. The landmark paper on this principle seems to be Donoho (2006), where it is shown that the compressibility of a signal (being sparse for some choice of coordinates) is a sufficient regularization criterion to faithfully reconstruct signals using a small number of samples. In a way, this technique has an element of Occam's Razor in it--it tries to find a solution, in some optimal basis, that needs the fewest non-zero numbers to agree with the measured data. At least, that's my take on it without having finished the paper.

The relevance of compressed sensing to image deconvolution is explored in Wiaux et al (2009), and it seems to be powerful. I'm excited by this deconvolution approach because it meshes well with the intuitive approach I've been taking to deconvolution, which was to use wavelets and a Markov Chain Monte Carlo optimizer to find the model with the fewest number of components that reproduces our data to within the noise. Compressed sensing seems to be exactly this idea, but is agnostic about the basis chosen, instead of mandating one like wavelets. Anyway, this technique may also be relevant to our foreground removal problem because we might be able to use it to construct the minimal foreground model implied by our data. For synchrotron emission, which should have smoothly varying spatial structure with frequency, I envision that this could construct a maximally smooth model that would allow us to use sparse frequency coverage to remove the foreground emission to the extent that it is possible to do so.

Is Tenure a Problem in Science Departments?

2009-05-04T07:33:00.000-07:00

Somewhat belatedly, I wanted to comment on the New York Times Op-Ed by Mark Taylor that addresses some of the flaws of the current academic system and proposes some solutions. The central problems that Taylor highlights in his article are that academic departments are too isolated from one another and from the world, and that there aren't enough academic positions for all of the people who are getting doctoral degrees these days. Taylor makes some very good points, but his article is strongly influenced by his experiences in a humanities department, and I am not sure how relevant his suggestions are for a science department.

Astronomy suffers from many of the same problems Taylor describes. There are far more graduate students and post-doctoral researchers than there are tenured professorial positions, and yet students are trained as if they were all to be professors. However, science students are often not paying their own tuition (it is paid by the grant of a supporting professor) and there are more options for science students outside of tenure-track positions because of the many sources of external funding that support scientific research. Unlike the humanities, there are a variety of scientific programming and research positions for graduates who are not seeking professorial positions. These positions are aligned with the education students receive through their doctoral research.

This isn't to say that there is not a major problem in scientific disciplines concerning the ratio of student positions to professional positions. Rather, it is that the problem may not be as closely tied to tenure and the longevity of tenured professors as in the humanities. The problem may be that professional positions available to graduating students are being occupied by the students themselves. Scientific research in the United States relies on a large pool of skilled labor. Currently, this labor is being bought at well under market price in the form of cheap graduate student researchers. If more of these positions were filled by full-time research professionals, we might have a healthier employment system for scientific academia.

The problem is that this raises the price of research in the United States and may result in a reduction in the total number of projects (and therefore, researchers) that can be supported. From the perspective of researchers, this may be a healthier state of affairs--to not be misled into spending 5-7 years underpaid as a graduate student only to find that the only way to continue to do what you've been trained to do is to continue to be underpaid. But unlike in the humanities, science graduate students usually have not accumulated debt beyond their undergraduate education and they have been supported (however cheaply) through this process. The solution, then, may simply be to ensure that prospective graduate students in science are well-informed about what employment prospects they should expect after they file their dissertation.

Interferometry File Formats

2009-04-20T07:17:00.000-07:00

When astronomers talk about software done right, they often hold up FITS as the gold standard. I'll admit, FITS has done more to live up to its namesake (Flexible Image Transport System) than many believed possible. But unfortunately, there can be too much of a good thing. Sometimes too much emphasis is put on defining an end-all-be-all file format, when all we really need are good tools for converting between file formats.

Data formats are a problem in radio astronomy software. Currently, there are at least three major formats (MIRIAD, UVFITS, and MeasurementSets), each linked to a major software package (MIRIAD, AIPS, and CASA), with rudimentary/non-existant tools for converting between them. Many have taken this current state of affairs as a sign that multiple file formats are bad and that the community should decide on a single format. Since each format is intimately tied to major software package, this battle over file formats has escalated to a war between software packages.

The mistake made here was blaming the file formats. File formats are not the problem. The problem is that the software for reading them has not been circulated in easily accessible modules. I am encountering this problem as I'm trying to get AIPY to be agnostic about file formats by wrapping them all into Python. Here's where I am:

The MIRIAD file format was actually easily wrapped up, owing to MIRIAD having a developed programmer's API.

MeasurementSets (with CASA) are giving me a lot more trouble. It seems that CASA, with all of it's C++ objects that are passed between functions, is something of an "all or nothing" deal. If I want to read a MeasurementSet, I apparently need to wrap up the entirety of CASA. The failing here is code modularity.

UVFITS is giving me the opposite problem.
UVFITS was cooked up as a FITS-conforming file format to handle raw interferometric data. Unfortunately, interferometric data isn't in picture form yet, so an extension of the FITS format (the binary table) was cooked up to accommodate that (Cotton et al. 1995). The result was a file format that is so general that it does not tell the programmer what the data actually means.

File formats exist to support the needs of different applications. They've been created out of need, and should not be dismissed as unnecessary. I recommend to the radio astronomy software community that we embrace these file formats and work on modular code so that they are accessible from any software package.

PAPER: 8 Station Results

2009-04-16T06:26:00.000-07:00

Yesterday I submitted our paper on the 4- and 8-antenna deployments of PAPER to the Astronomical Journal & astro-ph. Although it does not place any meaningful constraints on cosmic reionization (when light from the first stars broke up the bulk of the hydrogen gas that had been sitting around since it originally formed), it nonetheless illustrates a first level of calibration and analysis towards that goal. This paper should have some impact in the community, as it shows that we are fairly well along in our experiment and it provides a first look at some of the astrophysical foregrounds that will interfere with detecting reionization. This paper also will double as the last chapter of my dissertation, so publishing it puts me in the endgame of my doctoral studies... This fall I'm going to take an NSF postdoc back at Berkeley. Basically, that just means I'll be doing the same research (along with some teaching), but with better pay.

More on MCMC in Python

2009-04-14T06:04:00.000-07:00

Markov-Chain Monte Carlo (MCMC) seems to be a promising technique for the calibration/imaging problem that we are facing with our experiment the Precision Array for Probing the Epoch of Reionization (PAPER). Yesterday, in addition to taking a crash-course in MCMC, I also started playing with PyMC, which implements, among other things, MCMC using Metropolis-Hastings chains. A first shot at a simple fitter using PyMC went something like this:


import pymc, numpy as n, pylab as p
from pymc import deterministic

x = n.arange(-10., 10, .01)

def actual_func(a, b, c): return a*x**2 + b*x + c

sig = .1
tau = 1/sig**2
noise = n.random.normal(0, scale=sig, size=x.size)
mdata = actual_func(1., 2., 3.)
mdata += noise

a_fit = pymc.Uniform('a', 0., 2.)
b_fit = pymc.Uniform('b', 1., 3.)
c_fit = pymc.Uniform('c', 2., 4.)
tau_fit = pymc.Uniform('tau', tau/3, 3*tau)

@deterministic
def func(a=a_fit, b=b_fit, c=c_fit): return actual_func(a, b, c)

mdata_var = pymc.Normal('mdata', mu=func, tau=tau_fit,
    value=mdata, observed=True)

mdl = pymc.Model([a_fit, b_fit, c_fit, d_fit, mdata_var, tau_fit])
mc = pymc.MCMC(mdl)
mc.sample(iter=1000,burn=250)  

a_hist, a_edges = n.histogram(a_fit.trace(), 40, (0,2))
a_bins = (a_edges[:-1] + a_edges[1:])/2
p.plot(a_bins, a_hist)

p.show()

I chose this example over the ones that came with PyMC because it was much closer to the kind of calibration problems we will be trying to solve with PAPER. An interesting point that came up early on was the reliance of MCMC on an estimate of noise levels in the data. I remember this from the Maximum Entropy Method (MEM) for deconvolution that I coded up in AIPY. An interesting technique, used here, is to actually include the noise level (tau_fit) as a variable that is fit for. This way you can specify a starting guess for noise, along with a confidence interval, and not have to pull a magic number out of a hat. In this example, the fitter does a good job of accurately determining noise levels. I think what happens in this code is that the current guess for the noise level is used as part of the calculation that determines the next state to jump to in the Markov chain, and that new state make include a revision of the noise level. This clearly might be instable for more complex systems, so I imagine some amount of care must be exercised in leaving noise as a free parameter.

Image Deconvolution with Markov-Chain Monte Carlo

2009-04-13T05:56:00.000-07:00

I've decided to morph this blog to be more about short updates pertaining to current ideas I'm thinking about, rather than the long-winded philosophical rants I've posted so far. Hopefully this might keep me more engaged as a blogger and maybe even help me keep better track of the things I'm working on. Starting up this blog again, by the way, is a shameless procrastination technique, since my dissertation is due in about 1 month, and all my writing energy should really be focused on that...

After attending an SKA Imaging Workshop in Socorro, NM a couple of weeks ago, I've developed an interest in Bayesian statistics and Markov-Chain Monte Carlo (MCMC) techniques as they pertain to interferometric imaging. Having never taken a stats course, I'm scrambling a little to absorb the vocabulary I need to understand papers written on the subject. Fortunately, in this era of wikipedia, getting up to speed isn't that hard. After reading wiki articles on MCMC, Markov Chains, and the Metropolis-Hastings algorithm, I dived into EVLA Memo 102, which talks about a first shot at using MCMC for image deconvolution.

The Maximum Entropy Method (MEM) is a classic deconvolution technique (one I've already reimplemented for AIPY), but I'd like to go a bit further down this road. According to the standard implementation (which I gleaned from reading Cornwell & Evans (1984) and Sault (1990)) this algorithm uses a steepest descent minimization technique based on the assumption of a nearly diagonal pixel covariance matrix (i.e. the convolution kernel is approximately a single pixel). While this is an effective computation-saving assumption, I found that for the data I was working with, this assumption lead to the fit diverging when I started imaging at finer resolutions.

I think MCMC, by not taking the steepest decent, might be able to employ the diagonality assumption more robustly. I also think it's high time that deconvolution algorithms make better use of priors. The spatial uniformity prior in MEM makes it powerful for deconvolving extended emission, while the brightest-pixel selection technique in CLEAN makes it effective for deconvolving point sources. There's no reason we can't build a prior explicitly for a deconvolution algorithm that tells it to prefer single strong point sources over many weaker point sources, but also tells it that when all else is equal, entropy should be maximized.

Artificial Intelligence, Language Recognition, and Babies

2008-10-04T12:46:00.000-07:00

As you may or may not know, I have son who was born 6 (almost 7) months ago. He is just the most incredible thing. Besides excelling in the normal metrics of cuteness and snugglability, he is also doing something that all babies do, but is probably the most incredible of all--he is is learning. He is learning how to move, how to recognize patterns, how to read expressions, how to form phonemes, how parse sounds. He is training the most incredible neural network on the face of the earth to solve problems that the best minds in the world have been working on for decades and haven't come close to solving. And he's making it look easy.

How is he doing it? Why can a baby, who knows nothing of the world, who lacks a fundamental understanding of physics, biology, optics, machine learning, linguistics, language, categorization, (the list goes on...) succeed at these tasks when very bright people using supercomputers cannot? I have some theories--some from an intro linguistics class I once took, some from personal experience trying to code speech recognition, some from learning foreign languages, some from my signal processing background, and some from watching this little ball of wonder over here. For what it's worth, here are my thoughts on the matter.

First, some observations:

The problems listed above (motor control, image/speech recognition, etc.) are HARD. Just because the human brain is extremely adept at solving them, let's not make the mistake of underestimating their complexity.
Babies don't come out with the answers. They pretty much can't do anything in the beginning. On the other hand...
Babies have a definite propensity for arriving at a solution. They may not consciously know what they are doing, but they have a pre-programmed "boot sequence" that gets them walking, talking, and causing trouble by age 2. This boot sequence is remarkably consistent between babies (no baby walks before babbling, etc.)
Babies have trainers (parents) who are instrumental in their development. However, babies work on problems of their choosing--a parent aids in language acquisition, for example, but cannot get a baby to start babbling before they come to it themselves. You can see this all the time when you watch kids. They have incredible attention spans for the skills they are working on, but things outside of that range are summarily ignored.
Babies do not reason their way to solutions. Reasoning comes later.
Changing gears... large neural networks don't work. Small neural networks are very good at discriminating between patterns on a few set of inputs, but one can't throw 1e4 pixels into a neural network and expect to train it to recognize any old picture of a cat.
A lot of skills that we think of as a single skill (say, speech recognition) are actually many interrelated skills. For example, it is certainly my experience in learning foreign languages that: a) without adequate vocabulary, I have trouble hearing the sounds that are being spoken, b) without an understanding of what a conversation is about at a high level, I have trouble knowing what words to expect, and c) without being able to hear the sounds that are being spoken, I have no clue what a conversation is about. I'm not just being silly here. There's a real, circular dependence to speech recognition that requires several skills to be developed in parallel (phoneme recognition, vocabulary, grammar, cultural expectations) in order to advance.
And finally, humans have an incredible knack for categorization--grouping things by common traits, and defining groups at all sorts of levels of generality.

I believe the above observations are only consist with the idea of a modular mind with a very strong hierarchy. In order to overcome the fact that large neural networks are untrainable, the brain has to be divided into modules that trained at particular sub-tasks that require fewer inputs. The mere fact that the brain is built of neurons and that these neurons only have several inputs suggests that this must be so.

Furthermore, the division of the brain into these modules must be pre-programmed. CAT scans reveal that the same physical locations in everyone's brain are responsible for certain functions, and it's clear that reason, cognition, and other general-purpose processing in our brain are not primarily responsible for language, image recognition, or motor control (although in adults, sometimes skills like language acquisition and motor control are augmented by reasoning and cognition). Humans have a natural language instinct, and capacity for image processing and motor control that belies an underlying, inherent cerebral architecture addressing these skills.

So what is the upshot of all of this? I think that work on artificial intelligence needs to reflect strong modularity and hierarchy. While designing hierarchical processing is not hard, training the kind of multi-tiered, cross-linked, sometimes circularly dependent system that AI requires is. Why do babies go through the same boot sequence? Do the stages of child development reflect the trained of different tiers of neural networks in the brain's hierarchy? It seems possible to me that the wonder of the human brain might more than this incredible hard-wired signal processing architecture--a fundamental component might be this incredible boot sequence, taking 10-20 years to complete, that trains neural networks ranging from simple movement and stimulus response to abstract thought and language.

Attiyah and His Theories

2008-04-06T12:45:00.000-07:00

I guess when you're an astronomer, you have to expect to be the target of the occasional crackpot with their personal theory of the universe. My antagonist is Attiyah Zahdeh. I don't know where he's from (although devious research indicates he's on central time, so my current theory is Chicago), or how he got my email, but for a couple of years now I have been getting emails of which the following is the most recent example:

Attiyah's Planetary Motion

I introduce this hypothesis in order to be discussed by scientists. I do not claim that I now have any mathematical proof or practical model to support Attiyah's Planetary Motion. I consider that the Kepler's second and third laws themselves support my hypothesis. It seems to me that Kepler failed to conclude that, relative to the Sun, the motion of the planets is the same as of the pendulum.Thus, he coined his second and third laws as alternative statements to express the laws of the simple harmonic motion of the planets. I'm inclined to say that Kepler (1571-1630 A.D.) was not aware of the work of Galileo (1564.1642 A.D.) on the pendulum and the laws of its motion he discovered.

The hypothesis of Attiyah's Planetary Motion is four propositions:
1. The planets move not around the Sun but in front of it.
2. The planetary motion in front of the Sun is of the simple harmonic type.
3. The planet (the bob), gravitational force (the length, the line between the Earth's gravity center and the solar gravity center) and the Sun (the solar gravity center as the pivot point), altogether form a pendulum.
4. The planet oscillates in front of the Sun in a hemiellipse.

Notes:
1. This hypothesis is an alternative of Kepler's first law only.
2. This hypothesis does not apply to the motion of the satellites.

I have also been spammed with Attiyah's Sun Theory, which I think says that daylight is caused by charged particles (or X-rays, or whatever) hitting our atmosphere--much the same as the mechanism causing the northern and southern lights, and with Attiyah's Hologeomagnetosphere, which asserts that the northern and southern lights are generated by electrical currents in the earth's molten core turning our ionosphere into a giant CRT monitor.

I once thought that these were created as jokes--that "Attiyah" was just the psuedonym of a humorist. Dozens of emails (and several years) later, I'm convinced that Attiyah is real and in earnest. In fact, I've discovered that he visits his theories upon astronomy message boards with some regularity, where he has revealed complete ignorance about how the scientific process works by demanding that others attempt to disprove his theories (the burden of evidence is on the newcoming theory) and by flatly ignoring the evidence that was provided against them. Two years ago, I myself was duped into providing a detailed refutation of Attiyah's Sun Theory, only to have my response disappear into the abyss of cyberspace.

But I'm not bitter. In some ways, Attiyah's doing a lot for science education by getting amateur scientists to review how we know what we know--reminding everyone that the reason we have such a widely adopted set of theories is that they are testably confirmed and mutually consistent. If only intelligent design, creationism, and young-earth hypotheses met with half the ridicule that Attiyah's theories meet on the message boards. Attiyah's only mistake, really, was failing to incorporate a little theology into the mix.

Playing, and Why The Fast Track Wasn't the Best Track

2008-02-17T12:43:00.000-08:00

A NY Times article on playing today got me thinking about the indirect path I took to being an astronomer. I went to school at Harvard with a lot of very bright people (and even managed to marry one of them). The undergraduate academic experience at Harvard was a little hard on me, though it took me several years after I graduated to fully understand why. The first reason is pretty common to undergraduates at Harvard--intelligent and accomplished people who are used to being the best at what they do are suddenly brought into contact with quite a few people who are better than they are. For driven students, this blow to the ego can undercut some of the self-assurance necessary to work productively.

What was harder on me than turning in my "big fish" status as I moved to a larger pond was the cultural mismatch that existed between myself and the faculty with whom I came into contact. Harvard physics (or at least physics instruction) has a strongly theoretical bent to it, and while some modicum of application is maintained through the 2 lab courses we were required to take, I was always given the impression that applied fields were a cop-out for theoreticians who couldn't make the cut. The culture of disdain for experimentalists kept me on a theoretical track throughout college, long past the point at which I was "having fun". I can tell when I'm having fun, because I play. Playing, as defined in the article above, is "apparently purposeless activity." For me, that means trying to answer questions that aren't on the homework, just out of curiosity. It means starting projects, building things, and enjoying it. The farther I went down the theory track, I less I played with what I was learning. The undergraduate curriculum left little time for doing anything that wasn't strictly required, which was one problem, but the larger problem was that the path I was taking wasn't supporting the kind of playing I like to do.

I got lucky when I enrolled in an introductory electronics class with Paul Horowitz. I found myself, outside of class, trying to teach the computer I'd built to shoot a dart at a mechanical dinosaur. I modified a remote sensing, squawking penguin to spit water at passers-by. I didn't realize it at the time, but I was playing. After I graduated, somewhat at a loss for what to do, and burnt out with physics, I asked Paul if he knew anyone I could work for. He introduced me to Dan Werthimer at Berkeley, where I started designing and building electronics for SETI. Out of school, I suddenly had a lot more time for diversions, and I began learning Python and using it to write evolving programs that mutated their own source code. I tried writing speech recognition (I'll post someday about language acquisition, one of my favorite diversions). I made a guitar website to learn cgi programming. Most telling, I (mostly) gave up video games for computer programming, which indicates the degree to which this really was playing for me.

Eventually I stumbled into radio astronomy, where the physics that I learned (and really did love), met with the electronics and programming that I loved playing with. An incredible number of skills that I currently use were developed during my diversions, including Python programming, soldering, web programming, and signal processing. I never took classes in any of these things, I just learned them from my projects. What I didn't understand as an undergraduate was that working can really be "playing" if you find the right job, and that if you don't play with what you're doing, you might be barking up the wrong tree. Moreover, I was able to learn and accomplish much more when I was in a laboratory environment, playing with what I was learning, than in a classroom listening to lectures. Of course, you can't learn everything from playing--you need people to take you beyond what you have immediately at hand--but for me at least, I would rather this be the exception to the rule. Playing shouldn't just be for kids.

Rain Water

2008-01-19T12:42:00.000-08:00

I consider myself an environmentalist. I try to do the basic things around the house that help reduce my carbon footprint (you can calculate yours here) like installing compact florescent lights, installing power strips to prevent appliances like TVs from pulling current while "off", driving the minimum possible, eating vegetarian, and drinking soy milk instead of the dairy variety (cows are surprisingly bad for the environment). And I'm always on the look-out for something crazy and interesting to try for the environment. An idea I've been tossing around in my head (especially now that I live in rainy Puerto Rico) is harvesting power from rainwater. Part of my recent interest in rainwater, I must admit, has to do with the fact that we average about one water-outage per month here; I want backup water. However, my original interest, a year or two ago, was actually in generating electricity from the rain falling on my roof.

Here are some order-of-magnitude calculations for what I might expect to get from this. Quick research indicated that average annual rainfall here is 62 inches (about 150 cm). This means that each square meter of rooftop collects about:

150 (cm) x 100 (cm) x 100 (cm) x 1 (g/cm^3) / 1000 (g/kg) = 1500 kg

of water per year. If a story of a building is 5 m high, then the potential energy in that water is:

Force x Distance = Mass x Gravity x Distance = 1500 (kg) x 10 (m/s^2) x 5 (m) = 75,000 (J) = .02 (kWhr)

So PR generates .02 (kWhr/m^2/story) annually. My apartment building is about 10 (m) x 20 (m), and is 4 stories high, bring the energy of rainfall to 16 kWhr annually. If electricity costs $0.10 per kWhr, I could save a whopping $1.60 off my power bill annually. That probably won't ever offset what it would take to build the generator. Sigh.

Of course, dams do exactly the above, but with an enormous collecting area (not just a rooftop--an entire drainage basin). It looks like rain on my roof isn't going to power my computer, though. But I still might try to use the water to flush my toilet when the water goes off next weekend.

Hidden Variables

2008-01-09T12:41:00.000-08:00

I've been having a few conversations about hidden variables lately, so I thought I would post about it. First a little background:

Quantum mechanics (QM), as we know it, is weird. It is a classic example of science as a selection process, as I talked about in the previous post. It set about to solve the problem of predicting the locations, energies, and other microscopic attributes of fundamental particles. When the dust settled, we had one of the most accurate theories ever made (it predicts the mass of the electron to 14 decimal places), but the model used to make these predictions countermanded a lot of things we'd thought were true but never actually got around to testing--intuitive things like "a particle can only be one place at a time", well-established things like "no particle may carry information faster than the speed of light" (which is true, but can be violated over short distances), and most importantly for this post "if you had enough information, you could determine the outcome of any experiment." Nature's rejection of this last idea comes dangerously close to undermining the pillar of scientific philosophy that the universe is predictable insofar as it can be modeled and tested, and it offended a lot of scientists (including Einstein).

What QM says is that there are certain pieces of information that are not simultaneously knowable. If you know a particle's position perfectly accurately, then it is impossible to know its momentum. If you know a particle's orientation along one axis, you can't know it along any other axis. For a long time, many people (including Einstein) thought that this was a shortcoming of QM--that these particles have "actual, hidden values" that QM just didn't know how to predict, and eventually there would be a better theory that could tell us what these values are. Those hopes were shattered in 1964 when John S. Bell proved that there can exist no hidden variables in a way that is compatible with QM. His predictions have been validated by experiment, showing that the reason we don't know the state of a particle is because the universe hasn't made up its mind (excuse the anthropomorphification) until you measure the particle. Crazy.

I like to think of QM like a black curtain at a magic show that allows the universe (the magician) to perform all sorts of sleight-of-hand shielded from the eyes of the audience. On our side of the curtain, there are rules you have to follow--conservation of energy, speed of light, definity of location and state, etc. On the other side of the curtain, anything can happen, so long as when you pull it back out (when we make a measurement), the rules have been obeyed. The "curtain" idea isn't so far-fetched; it's an analogy to Feynman's well-tested theory that the outcome of an interaction is the sum over all possible interaction pathways. The universe takes advantage of this curtain to do things that we think should be impossible, like transmitting information about a measurement instantaneously between two particles that share a quantum state. Furthermore, the universe relies on the fact that we can never see behind the curtain (this is an interpretation of Bell's theorem) because if we could, we could use that machinery to transmit our own information faster than the speed of light, and that violates causality. Causality, by the way, is another principle that we cling to because it seems self-evident, but may in fact be wrong. It will take a unification of the theories of QM and general relativity to sort that out.

Shifting gears into philosophy, I was talking with my uncle, who came to visit last week, about how people look for meaning in their lives. His argument was that back in human history, when the universe seemed a jumble of arbitrary events, a physical, all-powerful deity with direct control over all that happened was a powerful metaphor for finding meaning in the events of ones life. However, as scientific knowledge has gradually encroached on the idea that a god can take direct physical action in the universe, religion has had to respond to the sense that science is pushing away meaning in life. My uncle's thoughts were that a physical deity is becoming an outdated way of looking for meaning in life, and that we need to think about a more spiritual, humanistic God. I agree with this philosophy, but I wonder if the rejection of the hidden variable hypothesis (the magician's curtain) provides a home for religions that require a physical deity.

Philosophy of Science (a Rant)

2007-12-18T12:38:00.000-08:00

I read an article today that got my back up a little. Let me give a little background to explain why.

I grew up in the rural, conservatively religious town of Rangely, CO. The first real experience I remember having that set me apart from the majority of the town came in 7th grade science class when we where discussing (you guessed it) evolution. Presented with another in the series of assertions that seems to constitute science education, I blithely bought in. My peers, who had been privately informed that evolution was not to be "believed in", discovered by waywardness, they dubbed me Monkey Boy. Yes, I know. Hilarious.

This was certainly not the last time that I butted heads with conservatives in my town, and it stands at the base of a growing sense of disconcert that I have felt towards organized religion. In the meantime, I went on to study physics in college, and am now a graduate student in astronomy (I do experimental cosmology). It has been surprising to me, now that I have started being introduced as an astronomer, to suddenly to have became an "Interpreter of the Voice of Science" (preacher) for people who want to know how our universe came to be.

The reason the above article got my back up was that I've always been careful, in these situations where I find myself describing the history (as we know it) of the universe, to stress the differences between science and religion. It's confusing terrain because cosmology has only recently come into the realm of science. My "what is science" speech usually ends up something like the following: science isn't a body of knowledge (despite what constitutes science education), or a belief system, or a religion. It is a methodology. Its groundings are philosophical, but that doesn't make its results philosophical. Science started from the philosophy that the universe is predictable: that one can predict the outcome of an experiment if one knows the initial conditions to sufficient accuracy.

Out of this philosophy (which the microscopic, quantum universe has helped us to understand is not strictly correct), a method was devised for arriving at an understanding of the predictability of the universe. Hypothesize, Predict, Experiment, Analyze, Conclude. Elementary school science fair stuff (except that "conclude" now means "peer review"), but still confusing. Science is a level playing field where anyone can make up any theory they want, in the face of all current knowledge if they want, using any strange force or being or Flying Spaghetti Monster, and science has nothing to say about it until you make predictions, devise an experiment to test them, and peer review to make sure it's reproducible. Science works like infants learning shapes--pick a block, try to jam it through the hole, and repeat until you succeed.

The framework that seems to work best for predicting the outcomes of experiments involves math. I'll save a discussion of why this might be for another blog, but it's important to realize that it didn't have to be this way. The universe could have chosen to play by different rules, or perhaps there are even models that can predict the outcomes of all the various experiments we've tried without using math. Math is just a model we have that works. Once you have two models that have equivalent predictive power, science has nothing to say again. Yes, I know about "Occam's Razor", but that's philosophy again, despite its popular portrayal as a pillar of science.

I get frustrated when people (even scientists and cosmologists) misunderstand science to be a body of knowledge, or a set of "transcendental laws", because (it seems to me) it's putting the cart before the horse. Whatever laws and rules we have are only as good as the degree of testing they have undergone, and are subject to revision and replacement as new experiments reveal their flaws. And we know they're flawed. The two most accurate physical models we have--gravity and quantum mechanics--are mutually inconsistent, and neither of them are able to account for an accelerating expansion of the universe (the "Dark Energy" problem) or for motions on galactic scales (the "Dark Matter" problem). So how can a self-respecting scientist hold up a "transcendental law" and claim it is more than our current best model?

My personal opinion is that referring to science as a body of knowledge makes it akin to religion--something transcendental and immutable (and inaccurate) handed down from above. What makes science science are the error bars: little reminders that our models are only as good as the extent to which they have been tested.