So much of the world seems to imagine that in each technology company, tucked somewhere, neatly in the corner of safe, there lies their cherished “intellectual property.”

The magic scroll, the secret formula, the special code, the transformative prototype — a boon which, if somehow, released to their competitors would instantly transform their lead into a trivial advantage.

Far be it from me to underestimate the value of an idea, but I think the error that most are making here is in underestimating the effort required in actually making something work. More than three years ago, I had the central ideas for what became LightSail Energy. Over that period of time, I and my extraordinarily talented colleagues have invested, collectively, the greatest efforts of our careers into developing the product and the understanding necessary to make it, a process that involved hundreds of experiments, thousands of decisions, and tens of thousands of tasks. We have almost two years ahead of us before our first product even ships.

One can not simply steal the kind of knowledge and expertise so developed. The momentum developed by one technical group cannot be simply transplanted into a competitor, it transcends documentation. No — it resides in our greatest assets: our people, the minds we’ve trained, the conventional wisdom we’ve transformed, the reputation that, through our trials, we have deeply entrenched.

While startups are growing, while their greatest advances and products are in their future, they need scarce not worry about a competitor stealing their work or ideas. Startups are in a race against time, not others.

It is even difficult to get people to steal your ideas. If only it were so easy to change the world.

Theft

This whole picture changes once a product is released, once made whole.

A novel may be simply transcribed, code copied. An engine, reverse engineered. Once there’s proof that something works, it’s easy for some to imagine that they could simply copy it.

But it takes much more than an opportunistic interest to bring most things to market. Even as Facebook began their long ascent, they feared that Google, or someone else who knew what they were doing, would just make their product. “And look how long it took them!” Mark Zuckerberg exclaims.

Google couldn’t simply steal Facebook, even though they knew how it worked, even though they had access to the clientside code. There was a barrier — the users and data of their growing population. And if one is going to simply copy something, one might as well try to improve upon it. The same urge to simply copy a work now becomes a stroke of inspiration.

The crucial factor? The common thief is lazy, and the lazy thief is thwarted. As you see, the thief is rarely a person of great motivation, excepting for personal vendettas. If other victims make better targets, one is safe. If all victims defend themselves more vigorously than the notion of honest work in the lazy thief’s mind, an honest society becomes an inevitability.

Theft != Transcription != Transformation != Inspiration

We must distinguish these concepts. There are simple semantic distinctions that our law and policy makers continuously evade. But they are incredibly important.

• Theft implies that the rightful person, an owner is denied access to something of value.
• Transcription is simply the copying of something, leaving the original intact.
• Transformation is the manipulation of a work into something of a different essential quality, message, or utility.
• Inspiration is a transformation less direct, acted upon through the medium of our imagination.

The great danger of laws that ignore these is not that they will prevent theft, but that they will so heavyhandedly prevent transformation and inspiration: the engines of our entire civilization.

Copyright has its merits, but most importantly, compared to patents, it induces limited collateral damage. Authors are protected, by property laws and window locks, from the most egregious of violations, theft, and by copyright, from commercial and transcriptions of their works, which might, it could be said, constitute theft of the market for their authorship.

Where copyright is dangerous is where it spreads. It spreads to non-commercial sharing of fragments, in music, criticism, and art, to the use and transformation of fragments. It spreads to the prevention of the dissemination of works to those who cannot pay, wouldn’t pay, could never pay. It prevents even the growth of the stature of the artist: it dramatically tips the balance of power of such industries away from the artist to those with the organizational resources to enforce their copyright monopoly on others: in music, the record labels, in movies, Hollywood, in science, Reed-Elsevier, Springer-Verlag. Their corporate lobbies will declare that they are protecting the artist, but in reality, the artist is humanized. The artist plays only a small role: second fiddle to the giant, thrumming machines of distribution, promotion, copyright enforcement, and market analysis in the publication industry’s leviathan mass.

What does this do?

Musicians, actors, filmmakers, and authors are enthralled to the callous calculations of multinational corporations, by structure insensitive to the local, cultural sensibilities that artists wish to convey. Those artists outside of a mature mass market industry where the promotion machine, defended by copyright, can create hits by bulldozing over works of artistic merit, are steadily seduced by those monied coffers: sell-out or be squeezed out. Indie artists are remarkable for their resilience and their art, but also for their poverty.

Remixes are prevented. The sampling of other’s work is believed to be theft. Internet services engaged in the promotion of new works are embargoed by those entities enforcing copyright. The remixing of footage of ten thousand films, which, as YouTube amply demonstrates, are deemed not inspiration nor transformation but the acts of criminals. And scientists, like me, outside of academia, outside of institutions which can mindlessly purchase the scientific journals of highest repute, are systematically shut out of the products of academic scientists, the works of public investment, which should rightly be the domain of everyone. Scientists are seduced by the well defended and financially supported reputations of journals in much the same way as artists are seduced by the distribution and glamor of the labels and studios, as models are seduced into posing and surrendering their image, for their glamor, their paltry salary, their many admiring eyes, their fame.

A better system, one could imagine, would draw the distinction between the theft of property, the theft of market share or opportunity, and the transcription or transformation implied by these other examples. A remix scarcely steals the market of the original work unless unreferenced. An immigrant entrepreneur scarcely steals the jobs of the natives unless they hire only immigrants as well — and even then this is unclear.

The heavyhanded application of copyright law is tantamount to the mislabeling of transcription or transformation as theft. If we are to grow as a knowledge economy, we must not commit such a grave error.

But all of these problems pale in comparison to the collateral damage done by the patent system.¹

Criminalizing Inspiration

It is a peculiar feature about a patented invention that it need not actually work.

It need not actually satisfy any needs.

It need not be, on its own, economically viable.

It need not ever been intended to be reduced to practice, nor spread out into the world.

It is an even more peculiar feature of patents that they do not grant you any rights, that is, other than that of taking away rights.

Rights to use of equipment that you own.

Rights to a methodology of medical practice.

Rights to manufacture or sale or application of an invention.

But most importantly, rights to inventions that you neither described nor anticipated, but that some aspect of your patent, another invention happens to incorporate.

Even if your patent discusses only the barest of sketches, and all of the hard work, and the vast majority of the good ideas necessary, were the result of other minds, whether independently, or by inspiration arising from the original work.

The patent system, then, makes a terrible sacrifice. Our physical property laws protect our stuff. Copyright, to a great extent, protects our creative, transcribable works. But patent law, in shoring up the defenses against these other violations, ‘protects’ us against, and criminalizes, inspiration.

But the patent system continues to grow. Business model patents. Medical patents. Use patents. Design patents. Continuations, and continuations in part. Nations even measure their inventive efforts by their cumulative accretion of patent applications. The scope of this heavyhanded mechanism continues unrelentingly, and unrepentant, chanting their mantra “We are protecting our ideas. Ideas have value.”

Ideas do have value. Great value. But the value of inspiration, of innovation, of allowing someone to make an improvement on an unfinished, or incompletely adapted idea, and bring it out into the world, is far greater.

If the ideas for stories could be patented, modern artists as great as J.R.R. Tolkien, George Lucas, and Steven Spielberg would have been sued as derivative.  The hero’s journey deeply underlies many of their works, in many forms. And who would have patented the love song?

SOPA – The Thermonuclear Option

The absolute misapprehension of these semantic differences, and the total disregard for collateral damage, in the past months reached a fever pitch with the introduction of the SOPA or Stop Online Piracy Act. It seemed as if everything that could be wrong with it, was.

True, as its proponents claim, it would give the corporate copyright and distribution monopolists one more tool to prevent sharing from degrading their dying business model.

But in a SOPA world, if one person shares one element and one corporation makes one complaint, then in one moment with zero due process and zero transparency, a website can be blocked, and the possibility for any transcription, transformation, or inspiration destroyed.

But not just for the offending material. For everything.

Share, once, the wrong content to Wikipedia, and the entire project, the greatest encyclopedia of all time, one of the greatest efforts of all of civilization, is threatened with extinction.

SOPA has been prevented — so far. But what halted process was that the ‘technical’ aspects of the internet confused our lawmakers. It is deeply disturbing that it was not the semantic distinctions between theft and inspiration, or the threat of inordinate collateral damage, that halted the efforts of SOPAs proponents. It makes one fear their judgments in other matters equally.

While it is tempting to make an analogy to our current middle eastern conflict, it would not, in truth, reflect our military operations adequately. The military aspired to surgical precision. Predator drones. Counter-insurgency tactics.

SOPA represents a different stance. To threaten wikipedia with destruction is to threaten to vaporize the nations thought to harbor Osama bin Laden. SOPA is absolutely the thermonuclear option. It, and the efforts behind it, must be stopped.

What this Means for Startups

Do not be threatened by others copying your idea. Do not even be threatened by others copying an unfinished product. They cannot copy you, nor the imagined futures in your head, nor the organization that you’ve built, nor the reputation you’ve gained.

Your job is to create something wonderful, get it out in the world, and make it so convenient and clear that you should be the one to buy from, that you should be the one to trust, that hardly anyone would attempt to compete with you. iTunes costs money, but is so superior an experience to Kazaa that hardly anyone would choose the latter.

Once you’ve released your product, your goal is to stay ahead of it. To improve it, refine it, and when the time comes, to supersede it — to have the success of your past project propel you into the next.

On rare occasions, a work or invention may operate, its works hidden, for the relevant time period of the interest of its creator. A high tech company might build their product in China, but integrate a single element, hard to make, hard to understand, at headquarters on american shores. A piece of software might require a special key; a chemical process an essential catalyst.

A business might hold a monopoly over these trade secrets for as long as they can, perhaps to wring continued business benefits out of it. This may provide some advantage.

But it will not last. At best, it will buy you time. And at worst, keeping secrets will hamper your own work; your story, your promotion, and all the internal communication of the company. Communication is hard enough when people are open and honest. Operating on need-to-know bases is torture — you don’t know what you need to know. Worst of all, it will keep you in the past — a cruel death to the innovative spirit, and a poor trade for a temporary advantage over a determined competitor.

Trade Trade Secrets

This risks of people discovering the secrets of your work are, frankly, almost always overstated, and the advantages of sharing, truly underrated.

We live in a global world. Interested, helpful parties can emerge from any of its corners. The more that you share, and the clearer that you make it, the further your reach. Helpful parties from any corner can bring gifts, information, criticism, or their own efforts. So much of what we now are at LightSail emerged from the people who over time approached us, fascinated by our mission.

We have secrets, of course. But it is impossible to track them all without hampering every conversation. So we will stay open. Not wide open — not exhibitionist — we can’t spend all of our time showing the world who we are and what we do, but open. We will let the conversation flow. And just as often as we share what we’re doing, people share amazing ideas of their own.

So don’t just keep trade secrets. Trade them.

Footnotes

1 – Notably, the one area in which patents are decently functional is the one where they are most similar to copyright: pharmaceutical patents. It is unambiguous whether a drug is chemically identical to another, just as is it unambiguous whether it is, despite a different printing process, the same book. Pharmaceutical patents are the exception that prove the rule.

We’ve launched our new website, and finally revealed the technology that we’ve developed and we think is going to change the world — regenerative air energy storage!

LightSail set out to prove that the science of our regenerative air energy storage concept works, and we have answered that challenge with a triumphant yes!

LightSail's Industrial Scale Prototype

We built an industrial scale machine by modifying a commercial natural gas compressor. We changed the cylinder head, added nozzles, replaced valves to allow reversibility, coated the surfaces to prevent corrosion, and threw our minds and hearts at the problem of showing that our approach could dramatically increase the efficiency of compressed air energy storage. Without water spray, and without burning natural gas, previous attempts at storing energy in compressed air topped out at less than 50% thermal efficiency — ok for a backup system, but not enough to change the world. This year, we aimed at greater than 80% thermal efficiency, at a high RPM (and therefore power), to show that unlike what people had assumed, high efficiency does not mean sacrificing performance.

We met or exceeded all our technical targets — demonstrating record breaking performance at the same time as record breaking thermodynamic efficiency — conclusively demonstrating our water spray heat transfer idea behind our regenerative air energy storage concept is effective at industrial scale.

Afterglow: the day I presented to Bill Gates

We presented to Bill Gates, a limited partner in the fund that invested in us. He was super excited by the potential of our project — that if we hit our targets it would change the world.

We spoke before hundreds of policy makers and energy executives, and helped instate groundbreaking legislation supporting energy storage in California.

Governor Jerry Brown

We have settled on our ultimate product architecture and design — a huge accomplishment. We’ve got a long way to go, but our models predict our experimental results within 5% RME accuracy, so we have some real confidence that it will hit all our hoped for technical specs.

We truly defined our market and value proposition. We’re aiming to make renewables plus energy storage a better and less expensive way to provide high value peak power than what the conventional sources — natural gas peakers, diesel gensets, and extra transmission wires – can muster.

The Trillion Dollar Formula

This is an utterly enormous market; at least a trillion dollars in size over the next couple decades.

IEA Estimates of Energy Infrastructure Investment Over 2008-2030. More than 30% could be economically addressed by renewables + energy storage

We’ve found that we’re uniquely positioned to reach that target, providing the lowest levelized cost of dispatchable electricity of any source, way ahead of our competitors.

But most of all, we’re excited about changing the world. Not only does energy storage make a renewables based grid possible, it also makes it economical. That’s the key to changing the world!

We’ve been working hard to uncover the greatest, most urgent opportunities for energy storage worldwide, and the opportunities we’ve turned up are simply massive. Energy storage is just what’s needed in places as diverse as Hawaii, Texas, Ireland, California, Paris, Denmark, Iceland, Nova Scotia, New York City, Australia, Chile, Dubai, India, and Subsaharan Africa. The scale and diversity of opportunities were astonishing. The most amazing thing? The willingness of governments to put their feet forward and most towards a future that’s right. We have been cynical; we believed that only once we had a full product, a long history, and economic parity under the most conservative of assumptions would governments move. We were proven wrong. Governments are leading the world into a clean future of energy. It’s utilities that are pushing back!

Steve the Redeemer

Take Iceland — a country of stark beauty. More than 80% of the country’s electricity is exported in the form of aluminum — the processing of which is one of the most energy intensive for any widely used material in the modern world. This single industry represents 40% of the Icelandic economy.

The Hellisheidi Geothermal Plant in Iceland's Golden Circle

Essentially 100% of their grid electricity comes from their amazing geothermal and hydroelectricity resources, and an enormous amount of their heating comes from geothermal cogeneration. Iceland is a land of abundant green energy.

Gullfoss -- the golden falls.

There’s a catch, though. Transmitting power across the sparse, weatherbeaten land is an expensive, unreliable proposition, where remote locations risk being knock completely off-grid with each storm. This is worse than it seems — if power is cut to aluminum smelters, the aluminum freezes, severely damaging the equipment. To backup the geothermal and hydro plants, then, industries have had to co-locate with diesel gensets — hardly a solution in light of the self reliance and environmental commitment of the Icelanders. We intend to replace these gensets completely. But we can do a lot more.

Low-temperature geothermal heat is available nearly everywhere in Iceland, and we can harness it. By expanding air at a higher temperature (and therefore pressure and volume) than when it was compressed, we get more mechanical energy out than we needed to compress it. This allows us to convert heat energy into mechanical energy, and from there, electricity. So instead of sitting idly like backup diesel gensets, our machines can be producing clean, geothermal energy, constantly; leaving the compressed air available for bursts of power when the grid fails.

So, we met with Iceland’s Minister of Energy — a former thermodynamics professor at Lund University, who bemoaned parliament’s inability to understand the concept of exergy.

We have therefore ‘rebranded’ our efforts. From now on, we have an initiative in ‘energy quality management.’ This they understand.

He understood the implications of an economical energy storage and geothermal electric generator immediately, and urged us to consider a project in Iceland. This is exactly the sort of progressive movement that governments are making and utilities resist. But we will overcome their skepticism! Stay tuned.

Catching My First Wave - A Good Omen

Of course, it wasn’t all business. If your mission call upon you to travel, it is your duty to truly experience the place. So I took the time harness some of nature’s forces myself. After I visited the grid operator and wind farms of Hawaii, I learned to surf!

We have continued to hire and improve our utterly world class team. We’re almost 30 people now, but I can tell you I have never before seen or even imagined such a diversity interests or depth of talent in a group. I work with the most amazing people I’ve met in my life! It is amazing to see how rapidly people are growing, but even more amazing to see how much more we can accomplish as a team. There are things that we literally couldn’t do on our own given all the time in the world — we have such a diverse set of skills in the company that we can make amazing things happen.

It was our first Burning Man. Our minds were blown. It is more than a festival, more than an amazing city. It is the most spiritually profound, unashamedly sensual, and maniacally creative place I’ve ever been.

Whiteout

Offering to the Sun

Deep Playa

We travelled as the chefs of the Airship Victoria last year; an airship project that eventually intends to hoist a Tesla-coil based lightning musical instrument. The camp, directly on esplanade, next to the flaming lotus girls, the sonic cannon, the flamethrower organ, and a 24 hour bar, was a surreal experience.

The Airship Victoria

It felt like… the future! It turns out, in the future, there are lots of lights, people float around on bikes, and jellyfish hover and flow.

Wonderful. But the main thing about the future is that people can’t help but be caught in the moment.

Stargate

Our camp featured tesla-coil concerts, and there, was, admittedly, high drama before the balloons were successfully fully deployed. Despite some initial setbacks, eventually the camp lifted their payload high into the air. In a city confined to a flat lakebed, the balloons added a third dimension to the playascape.

It’s impossible to describe the sense of flow one achieves in such a dizzying storm of self expression. We danced in drum circles in the nude, rode art cars and floated glowing jelly-fish, windsurfed and found inner peace. But what was most dazzling of all was the temple.

The Temple of Transition

A strikingly elegant wooden structure, built in just 10 weeks by inspired volunteers, the temple was a deeply spiritual place of reckoning. The visitors, pilgrims of every creed, came and prayed, and made offerings for their loved ones, those who that had left them, those who they had left behind. Poems, and pictures, incense and chants, old clothes or talismans, and cherished items of every description, laid respectfully to rest, ultimately fated to return to the atmosphere aflame.

Steve was so overcome that he bent down on one knee and made an offering to his mother, a brilliant opera singer, who left the world when far too young.

I miss you so much mom. You would have loved this place. I will love you forever.

As the temple’s towers, lean and graceful, slowly surrendered to the flames, glowing sparks rose deep from the inferno, and like wisps were carried up towards the heavens. The temple of transition, once a place of cool respite, now glowed brighter than the noonday sun. The crowd gasped as a shower of blue leapt out from the flames. Someone, days before, hid fireworks that launched streams of blue from the middle of the swirling firestorm, but in that splendid moment, it was impossible not to see those glowing blue apparitions, lifted high into the glowing sky, as souls, let finally free.

Meditation, Release, a Moment of Inner Peace

Upon our return, it seemed as if the whole of LightSail met us with faces silently asking us to bring them next year! We will.

This year, we’re starting a camp — tentatively named “Cleantech”. A solar powered shower and water recovery/purification system of our own design. It will be beautiful and efficient and environmentally friendly. Our kind of project!

LightSail's Firehouse Lab

At the end of the year, having wrapped up our work at our firehouse lab, having shown all we can with our current industrial scale prototype, we moved into our new facility — the former Scharfenberger chocolate factory, in which we will design, test , and manufacture our first product line. It is an amazing space. We will do outstanding work there, and we will be happy and proud.

As the move in completed, the holidays arrived, giving us the occasion to throw a lab-warming party for our friends and family. It was absolutely amazing. I felt as if the party unfolded as a microcosm of the entire project. It began with a simple idea: “let’s have a holiday party,” which lead to the conclusion “we clearly must have it at our new space,” and from that point, it took on a life of its own, spearheaded by people of admirable competence and outstanding creativity.

We were blown away by it all. The founders had no idea! Everywhere you looked there was perfectly executed brilliance.

Enter the space, we’re greeted with placards describing what all of the work is, what each of the rooms are, how each of the items work. There were demonstrations of our tank technology, our electronics and controls, our machineshop and quality assurance, our water spray lab, and even our original prototype (built in Ed’s garage using scrap parts and ebay!) We had no idea it would be there, and were blown away to see it!

Humble Beginnings: The Original LightSail Prototype, hydraulic, quirky, built of scrap, sweat, and parts ordered off ebay.

The original machine used a hydraulic approach — slower, with less power per unit mass or cost, and with higher inefficiencies, but we conclusively proved we could control the temperature of the air during operation, and control the valves to let only an amount of air in that would expand down to 1 atmosphere — yielding the very highest efficiencies. It was a cheap, quick way to show that some of our main ideas worked, and that we could build something. We sure have come a long way from that!

Travis O’Guin and his band played an incredible set of dixieland Jazz of some of the past century’s greatest compositions (ever wonder how “hit me baby one more time” is in dixieland jazz? Amazing.) Ed broke out into dance with a series of dancers, and the LightSail toddlers couldn’t resist the beat!

Machinemaster Todd Bowers breaking it down for Professor Robert Dibble and wife Helen

The machineshop was running — demonstrations included a CNC lathe disco ball, a hula dancer shaking it to an earthquake powered by the CNC mill, and just-in-time manufacturing of LightSail Branded Bottle Openers!

Dave Sprinkle spent years in the racing industry, but it's cupcakes that bring this smile to his face...

But what really stole the party were the cupcake cars, brought in by the brilliant Keith Johnson and his merry friends Lisa Pongrace and Greg Solberg. Our partygoers insist they’re even more fun to drive than a Tesla.

It was an unbelievable way to ring-in the new year. This is going to be a great one. That everyone injected such creativity and excellence in such a gathering just shows how much people care about their work and their team and this company and how high a standard they have for themselves. It seems as if everything at LightSail is like that — our people perform at a higher level than us founders can even think to ask of them, or indeed, even to imagine.

Energy Standout of the Year, Forbes 30 Under 30. As shot by the iconic Harry Benson.

To top it all off, we received coverage from none other than Forbes Magazine. I am honored to be highlighted as the standout in the field of energy in the Forbes 30 under 30 ranking! My extended family is finally less suspicious of my dropout ways. What a relief! I had a wonderful time at home with my brothers and little cousins and found to my amazement that my family had founded four businesses between us since we last visited. I guess it’s in the genes.

Christmas in Nova Scotia

This year looks to be even better. It feels like we’re reaching escape velocity.

I am honored to have been elected a mentor for the Thiel 20 under 20 Fellowship. These kids aren’t waiting to change the world, they’re just going out and doing it — I am so excited to be working with them!

I have been tapped to judge the Nova Scotia Cleantech Open, remarkable not only for being in my home province, but also for its amazing quality, rigor, and prize money ($100k free money with $200k of seed investment available.) I’m joining Matthew Nordan, of Venrock and Lux Research, whose work and judgement I have always admired greatly. His “The State of Cleantech VC is already a classic in the field.

We’re working full-speed on our product and technology, and are rallying allies across the planet to realize a wonderful number of as yet unannounced projects and partnerships.

Last year was an amazing year, but I have a feeling this one will be even better.

I am so happy to be alive at this moment in history. Great things are afoot. The winds are changing.

Jane: You “started” a PhD at Princeton… why did you stop? Was it boring? Did you feel driven to just do your own thing?

Danielle: I was more temperamentally suited towards my own thing – though that was only part of it. I entered a program in Plasma Physics to focus on fusion energy – the process that powers the sun. I thought that with a few good ideas, we could produce electricity more inexpensively than, say, planet-smothering, lung-blackening, mercury spewing coal.

Diptych - The JET Tokamak Fusion Reactor / Filled with Plasma

I guess I became disillusioned. The objective of fusion power is primarily to create a very inexpensive heat source. The reason people believed so strongly in fusion energy is that the fuel is practically free – unlike, say, coal. Unfortunately, we haven’t figured out how, even in principle, we might build a reactor that doesn’t wear out rapidly over time. Since you expend the reactor, it is most properly thought of as a kind of fuel. And if you consider the cost of ultra-high tech fusion reactors, versus pulverized coal, it looks like coal is going to be much cheaper.

I saw the technical work stretching out, endlessly before me, with no clear fix for our energy problem within sight. I got spooked.

There might be ways to solve these problems, but they certainly weren’t known.

I didn’t think the right things were getting funded. I saw my professors – brilliant scientists – spending most of their time in a struggle to acquire funding, rather than doing research. I thought there was a better way.

I came to Silicon Valley with the intention of making my fortune, and then funding research. After about nine months, I found myself compulsively doing energy research again, but this time, focusing on how to best harness and make use of energy from that great, fusion reactor in the sky, our sun.

Jane: On your Google profile, you call yourself an “ecopragmatist.” What does that philosophy mean on a day-day-basis, in your work and overall outlook?

Danielle: Environmentalism is a morality, or a philosophy. Ecopragmatics is a discipline.

Environmentalism, as a movement, achieves its greatest successes in raising awareness. Despite the fact that we live on this earth, cohabitate with nature and depend on nature for the air we breathe, the water we drink, the food we eat, the land we live on, for the microbes that pervade and defend and compose us, and despite the fact that being close to nature brings people joy, people have become strikingly unaware of their environment, of the flora and fauna within it. Environmentalists have made personal and tangible, the plight of ecosystems worldwide.

Where environmentalism fails is in its mythology; both in the mythology it has developed and promoted, and the mythology of greater society that it fails to deconstruct.

Environmentalists – and I realise I am over generalizing here — imagine that we human beings are apart from, and the scourge of, pristine nature, and that to save it, we must depart from it.

I do not believe human civilization is apart from nature in any crucial quality any different from an ant’s colony, a beaver’s dam, or a coral reef. Collective beings, using whatever tools and artifice at their disposal, have for millions of years altered their environments, and adapted to fantastically diverse environments. Our artifice is no different in kind; save perhaps, a degree of understanding and intention, and a means of development within generations, and a mechanism for sharing – language, to pass our tools between generations.

What is most astonishing about the rise of humanity is not its use of tools, not its use of language, not its environmental cultivation. It is the scope.

We have spread to almost every ecological niche of the planet. Humans consume one quarter of the enter base of the food chain, either directly or indirectly.

Less than one quarter of the world’s ice free land is wild, and only 20% of this is forests – wildlands account for only 10% of the primary production – or plant life – on the planet.

During the reign of our influence, 40% of the world’s phytoplankton, representing 20% of the world’s plant life, and 20% of the oxygen in the atmosphere, has disappeared over the past half century.

Under our influence the world has changed faster, more widely, and more profoundly than since phytoplankton drove the atmosphere from methane to oxygen. It is not that our changes are distinct from those of nature; there are plenty of creatures that change their environment far more that we change ours. But we have done so within a few hundred generations, at a planetary scale.

Pristine nature is a myth. Mankind has already profoundly changed the planet; every acre of it feels our climatic influence. We coexist. We are in nature, the question is how to live within it.

Environmentalism fails us here, because it stresses the difference of our influence, rather than its scope. Environmentalism is unwilling to consider small sacrifices which might undermine the purity of nature – hoping to preserve untouched ecologies from our desecrations, and yet even preserved land, as the famous Nature Conservancy has discovered, cannot escape our global influence. We worry about the little things; the spotted owl, the desert tortoise. And yet all around us, the world is undergoing – has undergone – a paradigm shift. As the great Paul MacCready states,

10,000 years ago, at the beginning of civilization, the human portion was less than one tenth of one percent. […] Humans, livestock and pets are now 97 percent of that integrated total mass on earth and all wild nature is three percent. We have won. The next generation doesn’t even have to worry about this game — it is over. And the biggest problem came the last 25 years: it went from 25 percent up to that 97 percent. And this really is a sobering picture in realizing we, humans, are in charge of life on earth, we’re like the capricious Gods of old Greek myths, kind of playing with life, and not a great deal of wisdom injected into it.

Nature vs Humans. Biomass of Wild Air and Land Vertebrates vs Human, Livestock, Pets

Ecopragmatism rejects the myth that the wild is endless, the oceans an infinite bounty. We recognize that the needs of mankind must coexist with the health of our planet, and that it will take sacrifices on both sides, from nature and civilization, to achieve peace – we advocate for both sides at the negotiating table. Ecopragmatism accepts that we must overcome ideology, and enforce discipline – that we must work hard, combine ecology with economy, and, as inventors and influencers, accept human nature, and create solutions that make the easy choice the right choice, allowing even the greediest or neediest of people to work toward a healthy world.

Ecopragmatism recognizes that we are now the earth’s gardeners. The earth is now shaped by our whims. We must accept the responsibility that comes with power, and consciously guide planet growth, with wisdom, courage, and decision, toward a thriving, sustainable future.

Jane: You’re the “Chief Science Officer” with LightSail Energy. What that involve? What’s a typical day for you (if there is such a thing as “typical”)?

Danielle: LightSail Energy is a green energy startup company that’s trying to make it possible, and economical, to power the world with nothing but clean, green energy. We’re tackling what some call the holy grail of green energy – how to economically and efficiency store energy such that intermittent renewables such as solar and wind can reliably and economically power our electrical grid.

Two days of output and wind speed from a four section wind farm. See the sudden drop? That's why we need energy storage if wind is to be a major part of our grid.

To do this, we’re taking compressed air, an elegant technology from a more civilized age, and using it to store energy. Compressed air is already considered to be the most inexpensive method for storing energy. Our objective is to make it more efficient.

When you compress air, what you’re really doing is converting mechanical energy into heat energy, inside the air. Conversely, when you expand air, you’re converting the heat energy in the air to mechanical energy. The amount of energy converted, for a given mass of air, is proportional to the temperature of the air.

The trouble starts when the air is compressed. To achieve a high energy density, compressed air energy storage systems compress to more than 100 times atmospheric pressure, or higher. In doing so, the air reaches extremely high temperatures, nearly 1000 C. This is too hot to manage practically, and so the air is compressed in stages, rejecting heat to the atmosphere after every stage.

By rejecting heat to the atmosphere, you lose the energy you’ve stored. Conventional systems add heat back by burning natural gas, but that still presents both a carbon footprint and inefficiency.

We have a different approach. By spraying water into the air during compression, most of the heat goes into the water, rather than the air. And water has a much higher capacity than air – 3300x at standard conditions. You don’t need to spray in much before what would have been a temperature rise of, say, 800 degrees C, becomes a temperature rise of around 20 C – much more manageable. We can then just hold on to the water, and the heat in a tank. We then spray the water back in during expansion, recovering the heat energy and converting it back into mechanical energy. We then convert the mechanical energy to electrical energy, using a motor generator.

There isn’t a very typical day at LightSail, but I often begin my day with coffee and breakfast with my cofounder, Steve Crane – a lapsed Caltech geophysicist who found himself first in the 3D graphics industry, and then the entertainment business, responsible for some of the biggest hits in history. He tried to hire me for a videogame startup; I ended up convincing him to join me as cofounder, and funding our first efforts through the sale of a house. We talk about everything. We talk about our technical challenges; the engineering and testing effort, the physics, we (try to) invent solutions to the challenge of the day, we try to figure out how to make our team as happy and effective as can be, we discuss the broader implications and applications of our technology, we talk about our philosophy, and duty as a company, and we talk about the world at large; everything from oceanography to filmmaking to dancing to the philosophy of science to the finer points of Italian cuisine. We’ve become best friends.

LightSail

Then it’s off to the lab – a converted historical Firehouse in Oakland’s Lake Merritt/Chinatown district. Our laboratories occupy the bottom two floors. In the bottommost floor we have a test cell with a control center behind bulletproof glass (just in case), a machine shop with a CNC-mill, and an assembly and quality control room. In the stable (the fire engine used to be drawn by horses) we have a laser lab for imaging sprays, and a conference room. The tower where they used to dry the canvas hoses has been converted into an exhaust manifold, outfitted with a muffler and a heat exchanger. We converted the hayloft to our electronics lab, and upstairs are our main offices.

We’re a little jam-packed in there. There are nearly 30 of us in total, so we’re moving to our new facilities (a 25,000 ft² former chocolate factory) in October. Until then, we’ve got our design room in the fire captain’s quarters, our experimentalists, electricians and technicians in the living room/kitchen/dining area, and our CTO and third cofounder, Ed Berlin, in a bedroom he’d converted to an electrical lab. The remainder are scattered throughout the machine shop, hayloft, and much of the analysis and business team (which includes Steve and I) reside in a penthouse in a second building across the street.

(I should mention that both Steve and Ed were prodigies in physics and engineering to the same extent that I was. Steve won his first research grant at 13, and entered MIT at 16; Ed, another MIT grad, built his first circuit at 3 and won the engineer of the year award from Grumman Aerospace – a 30,000 person company, mind you, at 21. People make a big fuss about me entering college at 12, but it was mostly that I had to find somewhere other than my dysfunctional middle school after dropping out. Plenty of other people could – and have – done it.)

The heart of the company is really the test cell, and we have a ‘driver’ from the racing industry (actually he focused on dynamometer tests, never racing in a car!) running most of the full-scale system tests. There’s a desk full of screens and controls, and one graph in particular, the pressure volume curve, draws particular interest from our theorists, and passersby.

If the test cell is the heart of the company, the whiteboard in the dining room area is the head! We often start discussions there, and people will drop in as they overhear and contribute. Our technical discussions can get quite intense, and we calculate much of what we need to make decisions by hand, in real-time, to verify the contentions and work that happens at our desks. All of our management comes from a deeply technical background, we all get our hands dirty, and we all dig into things and calculate them ourselves; especially with the most important technical decisions. There’s some replicated work, true, but we’re fast, and this gets us all on the same page. It helps to have generalists!

I’ll walk around the company, checking up on progress and issues, checking in to see if I can help people do their jobs, or if people are stuck, or need someone else to do something first to make progress. We spend a lot of time making sure people understand everything, and making sure that internal communication is handled correctly – it’s hard to get 30 people on the same page all the time! To that end, we get everyone lunch on Thursdays, which we have called “stupid question day.” Everyone is encouraged to ask their stupid questions – and people are obliged to answer them graciously!

Some days, we interview candidates. Our interviews are pretty comprehensive. We give a tour and the hiring manager introduces the position and the company. For most positions, we ask that people give a presentation on their previous work/explorations/education, and we ask lots of questions. Many people interview the candidate, and in the process we give both an in-person and a take home exam. The in-person exam involves much at the whiteboard, and sometimes we get people into the lab and have them solve an experimental puzzle and describe their solution. We have a world class team; the best that any of us have worked with.

I’ll return to my own desk at some point, and answer a slew of emails. Internal communication is critical! Then, every day, I’ve made a pact to myself to do at least one hardcore engineering thing – either in analysis, or making a spec, or designing a new experimental approach, or inventing something practical.

Maybe I’ll bike back, or grab dinner with Steve or friends. The evening and night are for big picture thoughts, and maybe a little music (I love my keyboard and guitar)

Then it’s to bed, and then, another day!

Jane: Would you have imagined five years ago that you’d be where you are now (career-wise)? Why or why not?

Danielle: I always thought I’d be working on my own projects, specifically in energy, but I fluctuated between thinking I’d do it within a university or my own company. It turned out that starting a company was right for me, at this time, with what I wanted to do!

Five years is a while ago now!

Honestly, despite the challenges and struggles I remember (among them, coughing my guts out before a critical presentation for us to raise our first bit of money during the financial crisis), I was surprised that it was this easy. Or maybe easy is the wrong term. I was surprised that it felt this right, at every step. There were no agonizing decisions. I went with my gut, and I am happy where I am.

Jane: What do you remember most about Dal’s Physics and Atmospheric Science program? What do you remember most about being at Dal? (i.e., “best” memories of both)

Danielle: I brought a friend of mine to the physics phyridays one summer. He couldn’t help but exclaim, while watching the players of hacky sacky, Frisbee, soccer, etc, watching the barbeque, and the liquid nitrogen shots, that the strangest thing about physicists was how *physical* we all were.

So those are my favourite memories: my friends and classmates camping, playing games, and being happy and young and brilliant and free.

I also fondly remember working with Jordan, my advisor, who really taught me a lot about how to *think* about science, and prowling around the labs, asking what people were doing, and naively offering to help.

Also – I loved the candy room. Whoever thought of putting a candy room right next to my office is an evil genius, and I offer kudos.

Jane: What was your biggest challenge while studying Physics and Atmospheric Science? How did you overcome it?

Danielle: The biggest challenge for me was balancing the wonder and curiosity that I had for the fundamentals of physics (and whatever question happened to interest me at the moment) and staying studious. You’d hope that doing one’s homework, and trying to get at fundamentals would be equivalent, but unfortunately not. Trying to understand why processes can be thermodynamically irreversible, yet are governed by fully reversible physics, isn’t something that helps you (much) in doing homework. Nonetheless, you need to do both to really grow as a scientist. I use the knowledge in from both my more disciplined physics training I received at Dal, and my personal investigations, every day. Dal gave me a lot of freedom to explore what I wanted to explore, but being the somewhat undisciplined person I am, I still found it hard to steadily “be a good student” when other inspirations stuck me!

Jane: Do you remember any particularly encouraging advice from profs, or any who were particularly inspiring because of their research interests or their engagement with teaching/students?

Danielle: There were so many! Jeff Dahn plucked me out of the giant Physics 1100 class and introduced me to physics research. His classes were also hilarious, which helped! My advisor Jordan Kyriakidis taught me so much of what I know about the scientific method, and rational thinking in general, and then set me loose on a bunch of incredibly interesting problems – some of which still bedevil me! Stephen Payne put up with my many questions about thermodynamics, and I credit much of what I finally understood – and what I put into practice at LightSail, to him. Andrew Rutenberg encouraged me to go to graduate school, and really got me thinking about my career path. There are so many more – I really enjoyed David Tindall’s astrophysics class, and the late Masoyoshi Senba taught me much about perseverance and rigor in solving scientific problems. But many of the best teachers were my classmates – we were a really tight knit group of kids and we learned so much from each other. We had a blast too!

Jane: How did the Physics and Atmospheric Science program prepare you for your current career? (either directly or indirectly…) Or, what do you find most satisfying about your current career? What’s the most challenging thing about it, and how has your education helped you?

Danielle: I haven’t really reached a point of satisfaction. I am like a traveler on a long journey, who knows the destination is yet beyond the horizon. But I know I am headed in a good direction – that I am on a good path.

The most challenging thing is when you worry that you’re off the right path! These existential questions are the most harrowing aspect of being an entrepreneur, or an inventor. We’ve made it through all of them though, so far.

Physics taught me how deeply one needed to dig in to something before one could say that one really understood it. How tenuous our knowledge was – and is – having our system of the world reconceived by each generation of scientists, over and over again. Physics taught me both to ignore the experts, accept my own fallibility, and to keep asking questions, to keep working!

Jane: What words of encouragement would you give students thinking of applying to the Physics and Atmospheric Science program, or students who are currently enrolled in it (especially those who might be questioning their choice of major)?

Danielle: Physics is like a bootcamp for your mind; I don’t think there exists another field of study that develops such powerful and versatile mental skills. It gives us a powerful lens with which we can ask questions of the universe, and of ourselves. Physicists can, and have, gone on to make major contributions in almost every field. Physics, as a field of study, gives you freedom.

Beyond this, physics is one of the most fascinating fields of study just by itself. It satisfies some of our curiosity, but then rewards us with still greater wonder. There are still a great many mysteries yet to be solved!

Jane: What’s your greatest accomplishment so far? (in any aspect of your life – from education to career…)

Danielle: Always looking ahead, yet having fun in the present! I don’t think about the past too much; what’s fun and satisfying is the journey.

I’m really proud of the work I’ve done as an entrepreneur at LightSail Energy, but I’m just getting started – it’s not an achievement yet!

Honestly, the two proudest moments in my life were dropping out of junior high and dropping out of graduate school. I’ll always remember to listen to myself if something’s just not feeling right – and I’ll always remember to strive to find something that does. I don’t know if that qualifies as an achievement, in the common idiom, but it’s something I’m proud of.

Jane: Where do you hope to be in 5 years? In 10?

Danielle: In five years, I hope to have caused the replacement of fossil fuels sufficient to power ten thousand people. In ten, I hope to have made renewables the economical choice for almost everywhere on our planet. And I hope to have helped hundreds of young entrepreneurs follow their dreams, and strike out, toward the unknown.

Skin Cancer

One of my first serious startup business efforts involved skin cancer: melanoma, specifically, by far the most malignant and dangerous type. It turns out that one can really do a pretty good job in terms of detecting melanoma in the early stages, when it’s still relatively easy to treat. This is in part because it is a cancer on the surface of the skin, and typically the cancerous, rapidly multiplying melanocytes produce an excess of melanin in patterns of characteristic irregularity. In other words, one can spot melanoma in ugly moles. If (a) one knows what to look for and (b) actually does look, in the vast majority of cases you can catch the disease before it spreads, and thus, save a life. The World Health Organization pegs melanoma deaths at 48,000 per year.¹

The ABCD's of Melanoma. Part of the ABCDs for detection of melanoma. On the left side from top to bottom: melanomas showing (A) Asymmetry, (B) a border that is uneven, ragged, or notched, (C) coloring of different shades of brown, black, or tan and (D) diameter that had changed in size. The normal moles on the right side do not have abnormal characteristics (no asymmetry, even border, even color, no change in diametry).

The chain breaks in both places. There are papers² that estimate the sensitivity of physicians to melanomas at 86%, which is not bad, if they happen to look, but not great either. The big problem is that it’s by no means certain that the doctor will look: most melanomas are still found first by the patient. That is, if they’re found at all before it’s too late.

A friend of mine refused to see a dermatologist, despite her friends urgings. Once she finally did, the cancer had grown out of control. She did survive, but she had to suffer through an extremely painful excision followed by reconstructive surgery followed by a recovery period nearly a year long. Prevention and early detection are the key weapons against disease.

This could have been prevented. There’s a heck of a lot you can learn from a photograph, even one taken from a cellphone. From that photograph, a trained individual can determine whether the mole is a cause for concern — whether you’re really just fine or whether one should be examined more closely by a specialist. There are even papers showing how an algorithm can be made to be as accurate as physicians in diagnosing the cancer.³

It seemed like a great hack. Within a week or so, I rallied some friends of mine, threw together an application to Y Combinator, and tried to put a business together.

Whatever the technical challenges were, I was unprepared for the legal and social challenges I was to face. The partners at Y Combinator did like the idea on some level, but declined to fund us. In Paul Graham’s words

“The trouble with the melanoma detection idea is that you’d spend most of your time dealing with legal and regulatory crap. That sort of work doesn’t really take advantage of your skills.”

There were other problems too (we were rather cavalier about the quality of images we’d get in practice from widely varying cameras, lenses and lighting conditions), but in the end it was the specter of law and regulation that cowed us. I still maintain that we could build this, and while perhaps it might not have been the absolute best focus for my efforts or launching point for my career, I still think we could have done a lot of good.⁴

What concerns me most isn’t that we couldn’t get the melanoma detection idea to take off. It’s that anyone trying to work in the field of medicine is going to have to slog through legal and regulatory crap. There’s a lot that can be done with simple little pieces of software and hardware to help people look after their health. Detection is the first line of defense. But as long as we persist in the current regulatory environment, where detection aids must disclaim whatever they imply or slog through the same lengthy and expensive FDA approval process regardless of their capacity for harm, entrepreneurs, engineers, doctors and scientists are going to be limited in the good that they can do, as well as the bad. Lesson one.

[JessNordell notes that both SpotCheck and SkinScan have taken up the charge! ]

The Electrical Grid

It doesn’t take too long before the familiar seems sane, and what we grow up with seems to have always been, bequeathed to us by ancestors blessed with the virtues of invention, vigor and foresight. I remember, as a kindergardener, when my mother took me to the local power plant, chimneys three hundred feet tall, how awed I was by the sights and sounds — the enormous burners and boilers, steam blasting through turbines powering vast generators, electrons pumping through transformers in gray substations of metal ringed insulators, transformers, cylinders, a geometrical landscape adorned with fins, cathedrals of wire, lines buzzing. Deep sounds metal clangs reverberated through the boiler room, amid hot steel, as my tiny feet traced halting paths along innumerable iron grates. Five years old, seeing this, I imagined the builders of these machines surely must have known what they were doing. Promethean gods walked the earth: behold! This was their handiwork.⁵

It shocked me to find out how the electrical grid really worked.

The grid, as it exists today, has almost no capacity to store energy. This means that at every instant in time, the amount of energy going on to the grid must almost exactly equal the amount of energy coming off. Otherwise, the grid rapidly fails in its operation, rising too high or too low in voltage, or shifting frequency, or succumbing to noise, spelling disaster to everything dependent upon a smooth, steady stream of power for its operation. That is to say, nearly every device of the modern world.

And while we have some provisions to redirect power on the grid (primarily, giant, motor driven switches of a kind not unlike those which control your household lightbulbs), our ability to throttle the amount of power we put onto the grid is slow and expensive. Gas turbines are the power plants that can be controlled most readily, however their power must be throttled over a period of nearly 15 minutes. When they do this, however, they operate away from their most efficient operating point, meaning that lots of fuel is burned, power is expensive, and more CO2 goes out the stack into the air. Hydropower stations can be throttled nearly as readily as gas turbines, but they can operate in a wider range before they lose most of their efficiency. Coal, petroleum, and nuclear plants, on the other hand require periods on the order of days to speed up and slow down — it takes about that long for them to relax to a thermal equilibrium, keeping thermal stresses within safe bounds.

However, the electrical demand is quite unpredictable, and it can vary significantly in a period much shorter than the 15 minutes it takes for our fastest power plants to significantly alter their power output. So to respond in faster timescales, power plants must run as ‘spinning reserves.’ This is where fuel is burning and the turbine is spinning, constantly, in expectation of a fluctuation. There are now massive power plants, burning fuel this very second, for no purpose other than to smooth out the rapid fluctuations on the grid. I hope I don’t need to point out how crazy this is.

What’s more, electrical supply from renewables is very unpredictable, especially whatever’s coming from distributed power. Wind power is the least expensive (in fact, according to Prof. Marc Jacobson of Stanford University, considerably less expensive than coal power!)⁶, but in terms of supplying controllable, usable power, wind is terrible! Wind power varies rapidly and widely, is extremely difficult to predict, is quite correlated over large regions, and, what’s probably the worst, wind power comes mostly in the dead of night, when nobody needs it.

Two days of output and wind speed from a four section wind farm. See the sudden drop? That's why we need energy storage if wind is to be a major part of our grid.

The entire world has wired itself up with an electrical grid that is fundamentally insane, and as we plug in renewables to solve the other problem with the electrical grid, controlling it is only getting harder.

We need a way to store energy. The current state-of-the-art⁸ is pumping water up a hill to store energy (essentially using a hydropower plant backwards), and letting it run down to retrieve it. This works well and is relatively inexpensive if you have the ideal geography, but those plant sites are mostly taken up: from now on pumped hydro energy storage is either going to be a lot more expensive or a lot more exotic (current plans call for giant undersea bladders, or the use of enormous underground aquifers).

Pumped Hydro Energy Storage

There are a number of other energy storage technologies, including electrochemical batteries, flywheels, and compressed air. There are good people working on every one of these approaches, and from our perspective, technically, there’s a good chance that at least one of us will find a solution allowing for energy storage so efficient and inexpensive that renewables, especially wind power, are economically competitive on the world scale, and on an unsubsidized basis.

However, among the many obstacles we find along the way appears the government. Despite their problems, technologies now exist that can both regulate the grid on short time scales (that is, dampening the second to second and minute to minute fluctuations) and, over a longer period, buy unneeded power in bulk at low prices and sell it when needed at high prices. If, on a single device, you can only provide the short or long term service, the economics don’t make sense. Luckily, it is technically feasible to do both simultaneously, on the same device, and thus the economics for the current technology can work out, in at some cases. The electrical grid can be buffered by energy storage, and utilities could make money doing it, to boot! How sensible.

Unfortunately, it’s illegal!⁷

Let me explain.

Transmission assets, like roads, railways, telephone networks and the electrical grid, have been shown to naturally tend towards a monopoly. In the case of the electrical grid, after a number of political fights the following deal was struck: the government would grant the electrical grid transmission monopoly to a single entity (in some cases more specifically the last few legs leading from the power plants to your home) in exchange for certain powers. In particular, the utility could, using its monopoly pricing power, impose practically any price on the unwitting public, which would pay just the same — they have almost no choice and electricity demand is notoriously inelastic. However, it would be restrained to charge a price only up to a certain regulated return on their capital, and no more. The regulatory commission has other powers, such as being able to define which sorts of investments and projects can proceed, and it places on the utility certain contractual demands to supply power.

Therein lies the first problem. In effect, were one selling power equipment for the grid, one’s real customer is not the utility, but the regulatory commission. It’s in the utility’s interest to buy the most capital intensive equipment that the regulatory commission will allow, and to incur the greatest expenses. After that, it may jack up prices such that it achieves its regulated rate of return, but now, due to its greater expense, it is now earning a greater profit as a greater entity for more investor dollars.

I never imagined such a perverse set of incentives. There are only three reasons for which utilities regulated in this way will look beyond the most expensive corner of the status quo. Firstly, in order to reduce risk on their capital investments, secondly, to explore the use of alternative technologies and broaden their strategic options, and finally, out of the goodness of their hearts. They’ve actually done a lot on that last count; that renewable energy is deployed at all seems to me shining evidence of a desire to do good. But utilities are conservative organizations, they’ve already figured out how to run their capital investments with very low risks (lower than most new technologies could possibly manage) and dollars for technology exploration are in short supply.

So, there’s the first shocker. Utilities with a regulated rate of return (I think this is most of them, though I’d love a better quantification for this) have essentially no incentive to save money, and hence no incentive to try out new money saving technology. If we’re to sell to anyone we have to sell to the regulatory boards, who are even more conservative. Oy.

Secondly, as the government is fond of doing, parts of the electrical utility business have been broken up, and they are not even allowed to talk with one another and share information, much less operate with the same hardware. The transmission department at PG&E is kept sealed off from the generation department at PG&E. Unfortunately, despite the fact that it is ludicrous, that it is patently insane to operate an electrical grid without energy storage (just as it’s insane to operate a network with no buffer, a computer with no cache or memory), there is no energy storage department at PG&E, or at least it doesn’t fit into the existing regulatory structure. There are no requirements for a certain amount of energy storage, and even the things that you could do to save money for the utility are unattractive for the reasons described above, and even those are unlikely with current technology to be good investments because they are both risky and it is illegal to operate the same device for both the generation department (people are saying now that storing cheap energy at night and selling it at a higher price during the day is called ‘generation’ for some reason) and the transmission department (regulating short term fluctuations in the grid and storing energy to ease congestion is called ‘transmission’ for some reason). So, a sensible and practically necessary thing (energy storage) is not allowed to be run in a manner which with existing technology which could save the public and the utility money and could enable the economic competitiveness of renewables. But that doesn’t matter because utilities aren’t trying to save money anyway.

Wow.

A couple of these problems can be avoided in the new, deregulated, ‘ancillary services’ markets. These deregulated markets, humorously enough, are markets for regulation — utilities and electric co-operatives pay a premium for MW scale power slices, so as long as you’re in the right spot and can deliver power on extremely short notice (people are pushing this as short as 4 seconds) you can stand to make a fair bit of money on a deregulated market selling to utilities. There are a few companies (Beacon Power is a prominent one) that try to make systems to do this now: in addition to the technology they’ve developed they’ve put in an incredible amount of political effort to get utilities to open up and deregulate their (*snicker*) regulation markets.

Unfortunately, the economics are still somewhat marginal. Current energy storage technologies are really expensive compared with the wasteful throttling of fossil fuel power plants. Better technologies (we’re working on one) might be able to make a really substantial profit. And if you can do energy arbitrage (buying power low, selling it high), or do something else that’s useful with your energy storage device, you can make even more money. Unfortunately, as above, it’s illegal to sell regulation services and do energy arbitrage on the grid — you have to do something else. (If any investors are reading this, don’t worry, we’ve got a few tricks up our sleeves!)

Someday, by our actions and with the help of others, enough of these problems will be solved so that the electrical grid can be buffered and made sane, that the public will save money by using energy storage, and renewable energy will be free to take over the world.

Someday.

Until then, we’ve got a lot of work to do. Here I thought we were working on a complete, technical solution to solve a massive problem in the world. A straight shot. And what do we find? Legal and regulatory crap, my nemesis, as defined by Paul Graham.

Lesson two.

So it goes.

A Patent Puzzle

Inventing things is what I do. It’s an inseparable part of who I am. I can’t get three chapters into a book before I have an idea for a new technology, product, startup, or hack. Parties are brainstorming sessions. I pester romantic partners as we fall asleep.

Some of my ideas are good. Some, not so good. Those that pass all my filters probably each have a decent shot at solving a major problem in a way fundamentally better than what’s come before with an approach that’s within my eventual capability to execute. Maybe it’s a little better than 1 in 3. I can’t really know until I try. But because I have so many ideas, and possibly because of practice, they number in the hundreds. If they were they only idea I had, I could really see myself giving them a go. And succeeding. I am not at all unique in this.

One thing that helps tremendously is to be able to talk about one’s ideas. Maybe even bring another person in to help work on bringing them to life. Oftentimes one’s ideas are too much for one to handle at some time, or maybe one is missing some crucial missing piece that needs just the right person or the right piece of knowledge.

If good ideas are too plentiful to fully utilize by oneself, if the difficult part of profiting from a good idea is bringing them to life, and if being open about them aids in this process, then ideas should be free.

Or, at least, it should not make one unduly vulnerable to reveal one’s ideas. One should at least be free to let one’s ideas free.

My friend, let me tell you about the patent system.

If I hold a patent on something (a widget), it does not allow me to build, license or sell the patented widget. It merely allows me to prevent others from building, licensing, or selling that widget (or anything else that falls under the widget classes and widget constituents for patented widget uses under the widget claims).

If my widget involves some device (a sprocket) that someone else (Eve) has patented, Eve can prevent me from building, licensing, or selling my widget unless I purchase a license for Eve’s sprocket.

If my something absolutely requires some crummy little item (a doohickey) that I forgot to mention or didn’t quite yet invent in my papers, my blog posts, my patent applications or my brochures, the sum total of my disclosure to the public domain and the patent office, and Eve notices this, she can patent said doohickey and the rational use of her doohickey and extort an outrageous price from me before I can sell any widgets at all.

Herein lies the problem.

Race to the Sea!

The Race to the Sea of the First World War, 1914. While the war was still mobile, both sides attempted to find an open flank. This began a race that ended when it reached the sea. Before these lines were drawn, armies were mobile. Afterward both sides became mired in grueling siege trench warfare from nearly unassailable positions. An analogy with patent strategy is apt: intellectual property is territory, and you do not want to be surrounded.

If you mention almost anything specific about what you’re doing and how you’re doing it, then you’re making yourself vulnerable. Eve can now take a look at what you’ve done, run through possible improvements, and possible requirements that you either have not claimed or disclosed or have not invented yet, and patent the ones that will hurt the most. Once you’ve given away your position, she can surround you, choking you off from the possibility of profiting from your work.

The important patent on the sewing machine was the sewing pin. With a tiny little hoop through the head. It is easy to forget the little things.

In the current legal climate, there are three options.

1. Stay hidden indefinitely. Protect everything that you’ve got by trade secret. If you personally (with or without corporate backing) expect to be able to bring your ideas to their full potential, this might be best for both you and the world.

2. Race to the sea. Patent your work — stake your ground, and reveal your position. Once your first patents are filed, you have to, urgently and carefully, check to make sure you’ve thought everything through. Are the relevant uses and markets patented? Are all of the incorporated or required devices available as commodities, or public domain, or inexpensively licensable, or do you own those patents yourself? Are you absolutely sure that you haven’t left anything critical out — that there’s nothing that you still need to invent that someone else might patent first? Are substantial improvements and variants of your design included in the first patent? There is a relatively short delay (on the order of a year) between when you file a patent and it appears for publication. If any of the vulnerabilities describe above exist, you’ve got until the publication date to protect yourself before Eve can make her attack. The clock is ticking.

3. Use your judgment, talk and write as freely as you desire, and cross your fingers. Hope that no one patents around you, and no one litigates.

I personally would love to be able to open source many of my patentable ideas. Engines. Desalination plants. Solar collectors. Refrigerators. Waste water mining. Wind turbines. Boats. Planes. A crowd beacon. Medical devices. Heat lamps. Translucent coatings. Metallurgical processes. Bioreactors. You name it. But the legal climate does not make it very safe for me to do so. If I write about them too freely, Eve might patent around me — now neither I nor anyone else can pursue my idea without encumbrance. If I protect myself by racing to the sea, I need to engage in an extraordinary investment of time, energy, and money on an idea that I probably crystallized in three seconds. It’s clear that even in the best of circumstances I’d be rate limited, but for almost all practical purposes this would become an untenable demand on my time (unless I have the resources of Nathan Myhrvold at my disposal.) Finally, I can protect myself by keeping my ideas trade secrets. Unless they are one of the few which I actually find the time and resources to seriously work on, this accomplishes exactly nothing. There is no pile of precious ideas I keep to stroke at night. I do not hoard them.

Some members of the free software community have been brainstorming ideas which have some relevance to tackling this problem. They include the Patent Commons and retaliation provisions in the GPL v3.⁹ Unfortunately they really do not provide full protection. Patents can only prevent people from building or selling things. They are weapons. Patent trolls, on the other hand, don’t build or sell anything. There is nothing to retaliate with, nothing to counter-sue, there is no center, no target. Patent trolls occupy an ecological niche in the legal landscape equivalent to cell based terrorist organizations in today’s political world. A sufficiently determined and evil person could extract billions of dollars from the world and destroy billions of dollars in wealth via long and painful lawsuits, simply by surrounding the patents and inventions that the world now relies upon. Perhaps at this time our only real protection is how grueling and soul destroying that path to fortune is versus the creation of wealth.

The patent system is supposed to encourage innovation. But I think what it’s done is cause everyone to overvalue ideas, and that has shifted the balance of power away from those who want to turn their ideas into something concrete and towards, marginally, those who have the ideas in the first place, but much more worrisomely, those who wish to wage war with them.

I’d love a discussion on how to fix this system. Specifically, not what to fix, but how: what specific actions can we take to make a difference? Quite apart from this, what should I do? Suppose I want nothing more than to do good for the world, to let my ideas, if they so deserve, blossom fully into life. Just talk about them and hope there’s no lawsuit? I do not want to be thinking about this!

Lesson three.

More legal and regulatory crap. Maybe I shouldn’t be so surprised: without law, there’s no formal property, no real capitalism, pretty much just anarchy, really. Of course it undergirds business. But what’s strange is that it’s constraining my thought. I just want to talk to people about my ideas, help the world, and be good. This shouldn’t be so hard!

So it goes.

Post Script: Trevor Blackwell of AnyBots and Y Combinator gives a thoughtful reply on Hacker News as to how the regulations that come into being are so malformed.

Big companies are smarter than gov’t regulators, they understand their business better, and they have a longer time horizon. So when the government comes around to regulate them, they think “OK, how can we turn this into a huge barrier to entry for new competitors?” They have large lobbyist and strategy budgets. They generally win.

The most misregulated industries in the US are energy, medical, and transportation. So there are lots of glaring inefficiencies, but they are there for a reason. Technologists assume the reason is stupidity and that clever inventions can fix things. Frustration ensues.

References

1 – Lucas, R. Global Burden of Disease of Solar Ultraviolet Radiation, Environmental Burden of Disease Series, July 25, 2006; No. 13. News release, World Health Organization

2 – Delays in diagnosis and melanoma prognosis (II): the role of doctors. Richard MA.

3 – Neural Network Diagnosis of Malignant Melanoma From Color Images, Ercal F. et al.

4 – I was told later by an entrepreneur formerly employed by the FDA that with a sufficiently awesome disclaimer one can put nearly anything on the market (“Nothing this program tells you means anything”) though I am really not sure how well this works in practice. I have read horror stories of the offices of a breast cancer detection aid being raided by the FBI — the case had to go up the the supreme court before they were acquitted.

5 – I do remember, however, challenging my tour guide in the waiting room when my cartoon host ‘Zappy the Electron’ claimed that both electricity and electrons traveled at the speed of light!

6 – Jacobson, M. Z., and G. M. Masters G. M., Exploiting Wind Versus Coal

7 – I should mention that I only know about these problems in the context of the USA regulatory framework. I’m sure there are analogous problems elsewhere, but I have not examined them in sufficient detail.

8 – State-of-the-art if one is considering cost effectiveness, and not energy density! Pumped hydro plants are gargantuan.

9 – This is a good discussion: Copyleft versus Patents: The Open Source Legal Battle, By François Lévêque and Yann Ménière

Let’s imagine the climate in question is not Earth’s, for a moment, and is instead the climate of a black box, hovering in a vacuum, with a big lightbulb shining next to it. Practically all its energy comes from the lightbulb (the rest from the residual heat within, and some dim source of central power), and practically all of its cooling consists in radiating infrared back outward. On the surface of this box tiny microbes are busy manufacturing and installing a layer of glass, which infrared cannot penetrate, covering it. We now wait, and see what happens.

The infrared is significantly absorbed by the glass, largely radiated back to the box, and thus the largest channel for cooling — essentially the only one capable of sustained cooling in the long term — has been attenuated.

Now replace the black box by Earth, the lightbulb by the sun, and the glass by CO2.

One would imagine the black box to have very strange properties were it not to heat at all. It might, for some time, somehow redirect some of the heat into less observable sections of its mass (e.g. the lower levels of the Earth’s oceans, which have a much greater heat capacity than its atmosphere). Yet this cannot last forever: there is only so much ocean. It might also become more reflective, absorbing less light (e.g. the earth’s clouds, desertification)? Yet an opposite effect comes from the melting snows and ice caps and constructed asphalt we add in urban areas: all of which have radiance and albedos observable from the outside (e.g. our satellites). Finally, the black box radiation is proportional to the fourth power of the temperature, so even if the percentage of radiated power that reaches the outside of the glass is diminished, if the temperature of the primary radiative bodies becomes less even, such that ∫T(new)^4 dA >> ∫T(old)^4 dA, the temperature can stay roughly constant. Other than that, there’s close to nothing that can be done: that box will very probably rise in temperature, and almost certainly the climate will change.

Skeptics correctly points out that the lightbulb varies in power output. And the black box is moving a bit relative to the light — further away or closer by — shinier or cooler or more black parts facing the light at any given time. They also point out that the glass isn’t the only thing surrounding the black box — for example they have noticed also a shiny layer of dust on the glass (aerosols), and an even bigger layer of glass underneath the glass we’d place (water vapor). And they point out that the layer of black paint appears to be, in a great proportion, liquid, and with a high heat capacity, and churning cyclically, and that there’s a lot of it, so that in any one instance a cooler or a warmer parcel of that liquid is showing.

None of this changes a thing about the fact that if we put yet another layer of glass on the box, the smart money is on it heating, and certainly on it changing. How could it not? At this point the onus is on these climate change skeptics to suggest a means by which the box is supposed to stay exactly the same.

Which brings up an interesting point. Maybe it is not so necessary that the Earth stays the same. Maybe there are credible arguments that explain that, really, the box won’t change that much, and for the teeming, glass manufacturing cultures of microbes living under the glass that these changes are not really such a big deal.

Some scientists, who I respect very much — Freeman Dyson for example, make this very argument. I respectfully disagree with him, as I think that there’s far too much risk in disrupting the biosphere, and that the disruption, famine, and loss of ecosystems and species that have already occurred are too great a price to pay, that oil wars, tyrannies, and people dying of respiratory illness from coal plants aren’t exactly positive either, and estimates of the probability of some catastrophic event happening, like say, Greenland melting, the consequences of which are too dire to imagine, range somewhere between 10% and 80%.

But that’s a philosophical disagreement. One might say that instead of engaging in ‘climate change’ skepticism, Dyson and others are engaging in ‘climate problem’ skepticism.

Too often, what we have with ‘skeptics’ is a scientific disagreement: the great majority say either that it is happening, but only as part of natural variation, and they had nothing to do with it, or that it isn’t happening at all. Which, at this point, seem more like the antics of a child screaming ‘I didn’t do it,’ or putting their hands to their ears, singing ‘la la la, I can’t hear you!’ than of a calm and reasoned scientist — or skeptic — examining the assumptions of a majority opinion. Their conclusions are already drawn.

—
PS:

For those already sold on the problem, my startup, LightSail Energy, Inc. is an exciting, well-funded startup in the $100 billion field of green tech energy storage. We are located in Oakland, California. We are seeking to fill several Mechanical Engineering positions. Applicants should have at least 5 years experience in product design of mechanical components.

Please be familiar with at least some of the following:

engine design, fluid dynamics (analytical, experimental, computational), heat transfer, thermodynamics, fluid power, pistons and seals, and multiphase phenomena.

Candidates should be comfortable with 100 kilowatt to multi-megawatt systems. Our needs range from mathematical modeling and design of experimental apparatus during the Research and Development phase, to designing for manufacturability and reliability. The ideal candidate will be a hands-on design engineer who possesses a high level of creativity and innovation required to be a valuable asset to the company.

Interested? Please send your resume to jobs@lightsailenergy.com.

Also known as H.E.H.

Furthermore, the lone autosave feature, hidden deep, is triggered upon every cell evaluation. Every time you evaluate something in a notebook, it will save the file – overwriting the last version.

I spent an hour looking to see what was the matter. I was convinced that it must be my own obtuseness causing me not the find or see or properly use the actual undo function there present. Instead, I found several newgroup threads in which people exclaim their amazement that such vital features as multiple undo and revision control hadn’t made it in, and apologists of various employ, Wolframites and others, who wrote back about the academic difficulty inherent in making multi-level notebook undos for notebooks fed with dynamic data.

I’ve got news for you Wolfram Research. It doesn’t matter. What people care about is their input data. What people don’t want about is destroyed work – the product of their labor, channeled through from their mind and out through their fingers, typed into little, nicely formatted cells. Almost everything else can be reconstructed from that input. The cases in which the inability of a simple multilevel undo to capture changes in dynamic data could possibly destroy work make up a fraction of a percent. Catastrophic hard disk failure is more likely.

I checked for a while to see if anyone found a solution more advanced than to manually save often, and use standard revision control. I couldn’t find anything.

Thus, I present my complete hack of a solution. It seems to work acceptably well — as in, I think I can rely on it to save my work, most of the time, if it really came down to it. But use at your own risk:

1. Get Dropbox. (getdropbox.com) This is a file sharing and revision control system that’s built right into the operating system, Windows, Mac OS X, Linux. One creates a folder, shares it over Dropbox, and every file in it is mirrored, automatically, on their servers. In particular, they keep track of revisions: so far as I can tell, every saved file is queued to be uploaded.

2. Enable Autosave. One can set this for the current notebook by evaluating the following line: SetOptions[SelectedNotebook[], NotebookAutoSave -> True]. This setting persists across saves.

3. Now every time one evaluates a cell, the notebook is saved. Further, while the file is apparently overwritten locally, the revision history (which I believe is, in most cases in full,) has been uploaded to Dropbox, or queued by the client.

4. To recover a revision, one can check the revision history in the Dropbox web interface.

5. Standard diff facilities are less useful, because the notebook files are highly structured. However, there’s a structured diff built into Mathematica, Notebook Diff in the AuthorTools package. One loads the package by evaluating << AuthorTools`. One can run this command NotebookDiff[nb1,nb2], where nb1 and nb2 are either notebook objects or their filenames with full paths.

This seems to work well enough for now. In the meantime, Wolfram Research, get your act together! You are driving people crazy.

I am also putting together a way to use the Units` package throughout (much more of) Mathematica, in particular, in conditionals and numerical functions. In practical engineering it could often be said that converting units is one’s primary occupation. For this reason the units package came to me with extreme promise. Built into the algebra package as it was, it could be used, in an extremely simple way, to check dimensional and unit consistency. However, it is not natively compatible with numerical functions: one cannot plot expressions including units, or expect results from comparison operators, and they can’t be used with functions like Max and Abs, or within the solver for numerical differential equations. It would be so wonderful if it could. I think I’ve figured out a way forward there, but we’ll see. The basic idea is to set to 1 all base units in a particular unit system — SI or CGS. This is likely to work for mechanics, however due to the differences in how the CGS and SI systems define electromagnetic quantities, making it work well for electromechanics is either a subtle project, or a doomed one. [Edit: David Park's ExtendUnits package appears to do exactly this. It's $30, though. http://home.comcast.net/~djmpark/UnitsPage.html]

Also, I don’t know if anyone’s really following this, but we’re not working on vehicles specifically anymore; LightSail is directing itself towards energy storage as a first product. Our website is woefully out of date. Vehicles or vehicle systems may be a further product and direction of ours, but by focusing on energy storage we can really prove out the core technology without the additional hassles of becoming an automotive company.

– M. Volkova

“I have a cold!” she said. “I am tired and chilled. I feel like nothing.” Hearing that, he took a small flask from his jacket. “Take a swill of this. You will feel better.” It burnt on the way down. “Tequila perhaps,” she thought, “if it is, it is the best I have ever tasted.”

She said nothing. With a smile, the man shifted his hat, turned his body, and walked away.

The next day, he visited my mother, sitting on the same porch. “Are you feeling better?” he asked, reaching for his flask. “Maybe a little,” replied my mother, quaffing the fiery liquid with uncertain haste. “Good,” he said. And with that, he pulled the collar of his jacket taught, smiled, and walked away.

In a blur the days passed. The old man visited the next morning, and the morning after that, and on and on for a week. Finally, he asked, “Are you feeling better?”

“I am!” she said. “Good,” he replied. He shouldered his bag and turned to go.

“Wait!” she faintly cried, “Won’t you let me sip once more of your medicine?”

“More tequila?” the man said concernedly, furrowing his brow. He waved his hands in circles towards her. “There is no need. You are cured.” And with that, he gave a smile and a wink, he turned, and walked away.

Dawn, a night and two weeks later. It was ready; the design for the both the engine and the drivetrain, encoded in a scattered handful of drawings and documents, one wiki, two heads, and a thousand lines of physical simulation code. The first test: powering a scooter through a staccato ride amid frenzied Manhattan traffic, calculating, by the hundredth second, the will of the engine, and the vehicle’s reply. We’d follow a path devised to track emissions from humming, throbbing combustion engines, byproducts of fuel burnt in tiny explosions sparked every second by the thousands.

But nothing save cool air would our machine exhale. Compressed air, ‘a thermomechanical battery’ of sorts, is cheap, long lasting, and quick to recharge (one need only open a valve, and if impatient, run a pump, the tank will fill in seconds.) What’s more, it’s efficient. A batteries charge begins life in mechanical form, in a spinning turbine if charged off the grid, or in the inertia of a vehicle, during regenerative braking. This is then converted into AC electrical current, which is converted into DC current, which, finally, is converted into mechanical energy, losing power at each step. To power the engine this whole process runs in reverse! But compressing or expanding air keeps mechanical energy mechanical (so long as temperature is kept reasonably constant.) In powering vehicles it is superior to the most advanced battery systems known. That is, in every parameter but one.

Historically, the low energy density of compressed air had crippled any attempt to venture further than a couple dozen miles; physics, it seemed, demanded tanks of excessive proportions to travel longer. At 300 bar (‘scuba pressure’), compressed air could release only half a percent as much energy as the same volume of gasoline burnt. We understood, however; it was an efficiency war. We knew that conventional vehicles were incredibly wasteful. There were many battles left to fight.

The Laws of Thermodynamics¹

“You can’t win.”
“You can’t break even.”
“You can’t give up.”

We hunted losses relentlessly. We were repaid with a series of compounding improvements, each building upon another, reversing the conventional patterns of efficiency losses endured by vehicles for more than a century. Finally, in a brilliant and unusually compact layout by Steve Crane, we found room to replace the paltry 1.3 gallon gas tank with one ten times its size. Nights yielded to our toil, and, slowly but surely our enemy routed.

“We’ve cracked the code,” we exclaimed. “The city is ours to conquer.” On the highway, whatever benefit earned by our scooter’s light weight, low rolling resistance and ultra-efficient regenerative braking would be dominated by air resistance.² But air resistance falls quickly with speed, and in the stop-start motion of the city our combined inventions would give our scooter an efficiency historically unmatched.

I keyed in the last few drivecycle parameters, drew a shallow breath, cocked my head, and pressed the enter key. The simulation lasted only a moment, but in that time, my little scooter ran more than one hundred and twenty miles, the equivalent of dozen rides between Wall Street and the Bronx on a single tank. “We’re in business,” I said. With that, and for all of a New York Minute, the questions, worries and restlessness retreated from our hearts. We huffed. “What’s next?”

[1]: To paraphrase C. P. Snow. Hat tip to Jonathan Smith.

[2]: Scooters are not particularly aerodynamic vehicles. Ordinary scooters have a drag coefficient of nearly 0.9, and a frontal area of 0.6 meters squared. We hope to achieve a drag coefficient of 0.6, similar to faired motorcycles ridden upright, but due to the rider’s position this will be difficult: some have described the aerodynamics of a scooter as like a “brick attached to a parachute.”

This may sound strange, but if you look at it a certain way, the introduction and popularization of tabbed browsing represents a tipping point in the history of computers. And, as they have altered and improved our ability to multitask, they represent crucial advances in the history of human thought.

There’s no fundamental difference between a tabbed web browser and an operating system taskbar. They are tools for multitasking. They are intended to make accessible each distinct thing which we seek to do, on our computer, so that we might easily glance at, remind ourselves of, or focus our attention on, whatever is most profitable, intriguing, or demanding of attention. Tabbed browsing popularized as web documents and applications overwhelmed their desktop counterparts. Computers had altered part of our working memory. Now, the web plays a leading role.

Computers have caused such an increase in our ability to multitask that they have precipitated what is one of the greatest generational shifts in patterns of thought since the scientific revolution. Quite literally, when we alter the way we multitask, we alter our minds.

Dangers

This is not without dangers, of course. We can still only keep so many things in mind (7 plus or minus 2, for most of us). Computers can display more tasks and more information than we can handle, and attention remains a finite resource. Computers are only a level two cache. If we don’t carefully control how we drink from this firehose, our attention may shatter, rendering us totally useless. For some, in today’s YouTube generation, this seems to have happened.

If tabbed browsing is a tool for thought, then we might expect that even small alterations in tab behavior will affect habits of mind. Consider the popular method of conducting research. One begins with a search, and lands on a Wikipedia page, or blog article. As one scrolls down the page, we middle click, opening tab after tab in the background. But where do they go? If the tabs land go directly to the back of the list, ordering tabs by age, we end up with something kind of strange. People tend to move to the next tab in the list, and so, the tendency is for our poor researcher to perform what is known in tech circles as a breadth-first-search.

There’s something to be said about this. Breadth first surfers only skim topics. They’ll cover many in one sitting, certainly. But they’ll tend not to learn in much detail. They will, in other words, learn many sides of many things, superficially. Shallowly. This could be one of the major reasons why, online, so many people talk so much about things they know so little about with zero apparent knowledge of this fact. Draw your own conclusions.

If, by contrast, tabs are opened directly behind the parent tab, we lure our researcher into a trap of a different sort. A depth-first-search.

Usain Bolt wins the 100m! Who’s he? Let’s check the Wikipedia… ah, it says here he was born in  Trelawny, interesting name — it’s a parish? Isn’t that like a religious thing? Hmm, it can be, but maybe it’s a civil parish, which is, hmm, a subdivision of the United Kingdom? Huh? Oh — I guess it’s part of the Commonwealth. What’s with that name anyway… seems kind of random… for “the promotion of democracy, human rights, good governance, the rule of law, individual liberty, egalitarianism, free trade, multilateralism, and world peace“, I guess those are good, especially… individual liberty… heh… I wonder how many times Dubya has trumpeted that one… Dubya… Maybe this has a list of Bushisms… which are apparently malapropisms… there’s a mouthful… where was I?

It’s easy to joke about, but it’s no laughing matter. It happens to the best of us. It wastes our time.

A Personal Note

People don’t always read straight through the tab they have open, and move onto the next. They zip around. Or they’d like to. Or, at least, I like to. When tabs first came out, I used to fill my browser with bajillions of them. You’d barely be able to click on one, much less recognize one by its hidden icon or title. There were tabs a pixel wide. Less, if I could get away with it!¹

So what did I do when I wanted to switch back to, gmail, say? I couldn’t find the old gmail tab. That would be impossible. So, I’d open a new one!

Soon, I’d have a zillion gmails, which would slow my computer to a crawl. And the tabs would be almost completely useless: it was totally impossible to switch around, all I knew if that there a lot of articles I wanted to read in there, and if I just kept working on the pile, then eventually the titles would see the light of day and I could use my tabs again. Usually I just crashed the browser. Oy.

Firefox 3 changed a lot of that, but not entirely for the better. Instead of trying to squeeze fifty tabs into one space, it just hides them offscreen. But the same problems come up: I still opened googles of gmails, and my browser would slow to a crawl. Like cold molasses, running up a hill. Honestly.

I’ve learned a little since then, and I’ve made some improvements. Some are technological — little plugins, restoring my sanity. The first is ‘aging tabs‘, by Dao Gottwald. It’s a brilliant little Firefox plugin that makes tab headers fade with age, from light to dark. It helps by reminding you what you were reading most recently.

The second is Tabhunter. Those familiar with Quicksilver or Enso will recognize its function: you just press ctrl-alt-t, type part of the name or URI of the open tab you’re looking for (like ‘gm’ for gmail)², and a list of matching tabs is displayed. Press up or down and enter to select the tab, and you’re there. There are a few minor problems with the design I think; for one, it’s modal (a box pops up, and you need to hit enter to dismiss it), and it saves your old patterns instead of clearing the input, requiring more keystrokes than I’d appreciate, but overall, it’s been a huge boon for me. It makes it much easier to switch between tabs, and I love it.

Finally, I use fullscreen mode. I found having a list of tabs staring me straight in the face had the effect of luring me away, towards something else, whenever things got hard. Now, with fullscreen mode and Tabhunter, I can concentrate on my work or reading, and switch around, only to do something definite, when I need to. This changes everything.

Suddenly, keyboard shortcuts became best of friends: ctrl-1 shifts to the front of the list, ctrl-9 shifts to the back, ctrl-w closes the current tab, ctrl-shift-t opens the last one you closed (or the second, third, fourth last one, in order), ctrl-d bookmarks, ctrl-l focuses on the Awesome Bar, ctrl-k zips to the search bar. alt-left or alt-right to go backwards or forwards. It’s way faster. The first time I started to surf this way, I had nearly 60 tabs open. I just kept reading through, fully completing each task in front of me, dismissing it when done (closing by ctrl-w), moving onto the next item in the list. I was absolutely startled, in just an hour an a half, my browsing was complete. I almost couldn’t believe it.

This has changed my life. You might laugh, but too many times I’ve stayed up, dozens of tabs tormenting me, unable to close them, afraid of sleep, of losing mindstate. I browse faster, concentrate better, and learn more. And now I am free.

Concentration

The lesson, I think, is that computers hold great powers.³ They can remember what we need to do, and what we need to know, freeing our minds from needless memorization, allowing us to concentrate on the task at hand. Or, they can present to us an endless stream of distracting tasks, tweets, and you-tubes, shattering our attention, and any hope of getting work done.

With each distraction, my mind thrashed. Heisenthoughts collapsed. Mindstate was lost. And for what? It takes far too long to start upon challenging work. Frequent taskswitching rendered me barely capable of any challenging work at all.

With the tab window hidden, my world changed. No longer was I tempted endlessly by the idle promises of articles unskimmed, tweets unheard, posts missed. I simply had to concentrate on the thing in front of me, and judge it worthy of time and attention, or not. It worked like magic.

I stopped being distracted by thoughts that there might be something marginally better that I could be doing. Instead I just did it. When I was done with it, I closed the tab. It might be, much more closely, a breadth-first-search, but, damn it, it worked.

Troubles

The trouble with this approach, I found, was that I was literally following a breadth first search with my browsing, which was not always what I wanted. Often, Tabs would present themselves in exactly the wrong order. Yet I’d still be tempted to go through them.⁴

People keep tabs around. They use their tab system as a to-do, to-read, or to-check-list. They keep tabs around as applications — things like gmail, and Facebook, and Pandora, which you might as well be logged into all the time. But keep too many things around, and you create a massive cognitive load. By hiding my tab list, I lessened that cognitive load. I outsourced prioritization to my computer. But my computer wasn’t any good at it. I switched tasks less often, which allowed me to concentrate, which helped me read, and helped me work. But it probably didn’t direct me towards the best thing I could have been doing. I worked, but, in some sense, on the wrong thing.

Priorities

I explained this dilemma to my friend Joel, breadth-first/depth-first analysis and all. He said that I had a linked list of task, and what I needed, instead, was a priority queue.

Duh.

Okay. So that’s goal one. Tabs are tasks: make it easier for us to prioritize our tabs, and we make it easier to prioritize our work.

Memory

Tabs serve as a kind of memory. We use tabs as a way to remember things we intend to do. But what aging tabs shows us is that we can also use tabs to remember things that we’ve done: specifically, what we’ve recently been looking at. What else might we be able to show?

First, let’s give ourselves a great big space to work. Suppose, say, when we hold down ctrl-tab (quasimodally), a great big transparent tab display appears over our window. We can zoom in or out, and drag the display around. We can even give it momentum: physics, just like the iPhone. We can toss it around. On it, what might we show?

Let us channel Edward Tufte for a moment. Let us maximize information density. Let us minimize data ink.

A Paper Mockup for FoxGlide

We can show content. We can display a title, with the favicon, atop a thumbnail of the tab. This is much better than just showing the favicon, as screenshots are far more likely to be distinct, and inform the user of the nature of their content. We can also show which bookmark folders that tab has been placed in or which tags the tab has adopted.

We can show order. Lay the tabs out horizontally, in the order in which they were opened. (Just like the current default, in Firefox).

We can show time. Near the top of the screen, make a small ticker, and mark the horizontal axis with small, vertical ticks: by clocktime, at distant regular intervals, and with icons at every change in activity.

We can show link structure. Connect every parent by an arrow to their children.

We can show relatedness. If two tabs share a domain, if they commonly are switched between, or if they share a large amount of semantic content, draw a faint line between them, to be highlighted when the tab is selected. In this way, we can automatically cluster groups of tabs used for a shared task or activity, and remind the user of tabs habitually used together.

We can show history. Turn the ticker at the top into a lifestream. At each tick, denote the activity by a little icon. Suppose you’ve done a bunch of typing: this is a writing tab, we can represent this with a little ‘key’ icon. Suppose instead that you’ve just scrolled and read: show scroll buttons, up and down. Suppose we’re idle: show an ellipsis. Suppose we open a new tab: show a sprout. Suppose we open a tab in the background, creating another branch: show an arrow branching into two.

We can show age. Let the tabs fade with disuse, as in aging tabs.

We can show status. Make the title of as yet unviewed tabs blue, as in unvisited hyperlinks. Make the title of visited tabs purple. And instead of wiping closed tabs off the display, make the titles brown, and let them inhabit a ‘history gutter’ nestled quietly at the bottom of the screen — to view it more fully, pull-up. when a tab is closed, maintain the arrows, connected to parents and children, but fade them so that they do not distract. By keeping closed tabs accessible, we seamlessly integrate browser history with the tab display, enabling browsers to enhance our memory still further.

Finally, we can show priority. Let the vertical axis represent priority. Higher implies more pressing. Priority can be set automatically: Age lowers priority. Activity raises it. Or you can set it manually. You can drag each tab along the vertical axis to set the priority, or you can strike a number, to alter the priority of the tab currently viewed, or otherwise selected. We can even use a hybrid approach: after a while, an algorithm harness the implicit, ambient information from our browser use, to learn which sites, keywords, and activities you consider high priority, and set priority for every other site you visit accordingly.⁵

To make it easier to set the priority for large groups of tabs, the priority of related tabs, for example, parents and children, influence each other. This implies that, while raising the priority of one tab, the arrows connecting parents and children pull the related tabs up elastically. We can also uncover tab relatedness by whether they share a domain, or if we would like to get really fancy, we could even link tabs by shared usage of ‘statistically improbable phrases’, as Amazon might.

Navigation

With the tab display active, we can quickly type to find the tabs we want. Type ‘wiki’, and all the tabs matching ‘wiki’ are shown, filtering the others out, which fade away. Hit left or right to progress through the history of open tabs: in the same order as is currently used by Firefox. Hit up (or enter), and we switch to the the highest priority tab (that isn’t being currently viewed) matching the filter we typed. Hit down, and we instead decrease priority, stepwise.⁶ It’s like Tabhunter, but using less keystrokes.

The labs team has already come up with a decent navigation system. They suggest that we put navigation in the Awesome Bar. If someone goes to the Awesome Bar and types a pattern that matches an open tab, pressing enter will take them to it, while pressing option enter will open a new tab.

I worry about this a bit. As much as I like the Awesome Bar, keeping focus on it is modal. And you need to select it (ctrl-L, or a mouse click), and then type in the name, making it a bit more complicated.

Also, it messes with important prior behavior. Most people will expect to go to somewhere on the current tab when they hit enter on the awesome bar. Changing this behavior might confuse users, conflicting with previous program models, doing so in a way that does not make evident that program behavior has changed.

Finally, compared to the approach I describe above, it simply doesn’t show as much information. It’s not as useful as it could be (note: see the mozilla labs wikipage on visual navigation for more ideas.)

Status

Where might you be able to download this? You can’t, yet. It is, for now, just a twinkle in my eye. But we’re on the cusp of something. The Mozilla team is just now in the process of deciding what new tabbed browsing features they want to build into Firefox 3.1, and are soliciting ideas. If you like this concept, or think, at least, that some of the ideas are worth considering, or if you have your own ideas, please, get involved in the conversation!

Also, if you’re interested in helping to do some mockups or some Firefox extension coding, please let me know (email me at daniellefong at daniellefong dot com). It will stay a small, open, simple project; this won’t be my fulltime thing. But for me, at least, I cannot ignore the chance we have to deliver ourselves from cognitive overload. To free us to explore the web more fully. To enhance tools for thought, and thus enhance thought itself. As whimsical as that might sound, this hope shines far too bright for me to simply let go.

Notes:

[1] – In other words, without crashing the browser. Opera was much better in this regard than Mozilla.
[2] – This actually works with regular expressions too.
[3] – With great power comes great responsibility.
[4] – This was not the only problem. sometimes, I’d see a link, and I’d want to follow it, just a little ways, to see what was there. But if I opened it in the background, I’d have to finish all the other tabs just to get there. Ahh! Instead, I just followed the link without opening a new tab, with the hope that I’d remember to return to the parent page and alt-left backwards. Soon I found the hotkey for flipping to the end of the list (ctrl-9). That helped a bit. I could open a tab, and flip to it in back, though I’d have to use Tabhunter to get back where I started.
[5] – It’s *Xobni for Firefox*! The web is the killer app! (I promised myself I’d never say things like this, but I actually kinda like it.)
[6] – Note that under this key scheme, ctrl-tab doesn’t switch to the next tab like it used to. That would now be done with ctrl-tab-right.

Thanks

to Alex Lang, Nick Pilon, and Joseph Perla for reading drafts of this, Sasha and Patrick from BaseShield, for the encouragement, the Mozilla Team, for running this wonderfully open design process, Blacktree and Humanized, for showing me how good navigation could be, Jenny Boriss, for sparking the tabs discussion, and Joel Muzzerall, for the inspiration.

PS

I’ve given some thought to the name. I, personally, like FoxGlide, which also exhibits the wonderful feature of my having parked it.

Please Leave Your Comments!

This post has sparked some interesting on Hacker News. What do you think? Does the design pass the Phone Test? Are the angles I haven’t considered? Are like features you’d like to see?

\ˈhī-z^ən-thȯt

alt. he-is-en-thought, as in, he is in thought, don’t disturb him… (thanks to Marc Chung @heisenthought)

noun.

1. A thought subject to mental collapse triggered by interruption.

2. A mental event or sequence characterized by coexistence of loosely related, sometimes contradictory, sets of knowledge or tasks. Highly dispersed in knowledge space, though it may support well defined momentum. Apt to tunnel through imposing technical or energetic barriers.

Collapse triggered upon the observation of the state of current knowledge by a third party. During the beginning states of observation, keen thinkers engage in a mad scramble to save mindstate, which may at some later time be partially reloaded. Status reports are heisenthought’s natural predator. Occasionally, members of a team will spontaneously ‘go dark’, in an attempt to think through some difficult problem — sometimes the only strategy which will work to extricate oneself from local extrema in solutionspace.^1,2 Caution should be taken when observing said people, as doing so will both collapse their progress into a single, likely suboptimal path, and be likely misinterpreted as disloyalty, hubris or laziness, since they have disengaged from discussion with no apparent progress.

3. A mental event or sequence characterized by firmly specified and demarcated generating bodies of knowledge, and ill discernible forward movement. Induced by overspecification of problem, method, or solution. Collapse upon observation of current direction of progress by a third party. “‘Are we moving forward on this?’ ‘The heisenthought collapsed; we’re moving, just not in the right direction…’”

4. A coherent collection of thoughts sufficiently isolated from random outside influence. Obeys the relation Δknowledge Δmomentum ≥  ħ/2.

5. A tongue-in-cheek term invented to cast off workplace ‘pings’, ‘check-ins’, ‘status-reports’ and ‘how’s it goings?’.

Notes:

[1] – For example, read the interviews with Max Levchin in chapter 1 and Blake Ross in chapter 29 of Jessica Livingston’s Founders at Work.

Max discusses how he went dark for a year, and came up with the something that really mattered — a way to fight fraud.

We had this merger with a company called X.com. It was a bit of a tough merger because the companies were really competitive—we were two large competitors in the same market. For a while, Peter took some time off. The guy who ran X.com became the CEO, and I remained the CTO. He was really into Windows, and I was really into Unix. So there was this bad blood for a while between the engineering teams. He was convinced that Windows was where it’s at and that we have to switch to Windows, but the platform that we used was, I thought, built really well and I wanted to keep it. I wanted to stay on Unix. [...]

I had this intern that I hired before the merger, and we thought, “We built all these cool Unix projects, but it’s kind of pointless now because they are going to scrap the platform. We might as well do something else.” So he and I decided we were going to find ourselves fun projects. [...]

It was me acting out, but it was kind of a low time for me because I was not happy with the way we were going. Part of having a CEO is that you can respectfully disagree, but you can resign if you don’t like it that much. But then eventually I became interested in the economics of PayPal and trying to see what’s going on in the back end, because I was getting distracted from code and technology. I realized that we were losing a lot more money in fraud than I thought we were. It was still early 2001. If you looked at the actual loss rates, they were fairly low. You could see that we were losing money, but, given the growth of the system and the growth of the fraud, fraud was not that big of a problem. It was less than 1 percent—it was really low. But then, if you looked at the rate of growth of fraud, you could see that, if you don’t stop it, it would become 5 percent, 10 percent of the system, which would have been prohibitive.

So I started freaking out over it, and this intern and I wrote all sorts of packages— very statistical stuff—to analyze “How did it happen; how do we lose money?” By the end of the summer, we thought, “The world is going to end any minute now.” It was obvious that we were really losing tons of money. By midsummer, it was already on a $10 million range per month and just very scary.

[2] – Blake discusses how, by necessity, they closed up the source and process during the early stages of work on Firefox.

Phoenix was basically a fork of the Mozilla code base that we controlled. We closed off access to the code, because we felt it was impossible to create anything consumer-oriented when you had a thousand Netscape people in search of revenue and a thousand open source geeks who shunned big business trying to reach consensus. We just wanted to close it off and do what we thought was the right thing. We went through a couple name changes, Mozilla offered us more support, and that’s kind of how it all got started.

Thanks to Marc Chung and Joel Muzzerall for reading drafts of this.

Our company is going to have graph paper placemats. Jealous? ;-)

organizations
relationships
communities

as living entities

as where no one cell holds the soul of a person
no one person, incarnate
may form all essence of a community
nor live throughout its lifetime
wholly part

as one cell expires
one soul retires
others, some new, fill their place

generations cycle
yet something remains
an living entity unto itself
though it may be true
that some people or ideas
resemble vital organs

the liver, to filter
the stomach, to process
the lung, to supply
the mind, to direct
the heart, to power

too often
transplanted,
perfectly good hearts
from one being to another
from one community to another
raise immune reactions
edicts like allergens
mobs like histamines

without disarmament
no transplant succeeds
no hire, restructuring, or revolution carried through
without somehow disabling
or surviving
societal antibodies

for some, it seems
while becoming part of a great organization
or movement
or organism
they enter through digestive tracts
to supply the raw stuff
the best
carbon, iron, calcium, oxygen
skills, goals, ideas, selves
are prepared
and cooked
chewed
dissolved
filtered
processed
and made, finally, into a part of something new
to fit some other master plan
a genetic blueprint
an ideology
a mission
a politic

for some
no greatness of which they are part
could come to redefine them
their raw stuff
the parts comprising
can find no better molecule
no further local minima
no structure more solid
between surrounding stomach walls
they are indigestible

or too delicate
too beautiful
too unique
to be eaten

their soul to realize
as part only whole
of something profound,
meaningful,
believed in
or as the kernel of a gem
of another body
of an essence of its own
starting anew

So prediction is fundamentally at the heart of a technologist’s work. At the highest level, we must predict to find what work focus on, and what future to aim for.

You might then think that prediction, as a skill, is worthy of practice. And practice it gets. In living rooms, in pubs and classrooms and yearbooks and dial-in talkshows and newspapers and blogs and comment threads and slashdot and every polluted corner of our existence, you find evidence: prediction is practiced all the time.

There’s a problem. In most areas of the technologist’s pursuit, it’s easy to see whether you’ve done well. Code should compile. Planes should fly. Cars should go. Bridges should stay up. We have a lot of honesty in our discipline, much of it because we are blessed with tests that we find hard to fool.

A typical test for predictions, on the other hand, is whether the story sounds good at the pub. You make some exclamation. People nod and clap. Everyone forgets.

This would be fine if you’re just looking for some conversation. But if you are, like technologists fundamentally in the business of creating the future, it becomes lot more troublesome. We are left to ignore predictive incompetence until reality slaps us coldly across the face. We are flying blind.

Taking a cue from Trevor Blackwell, I’ve decided to inject some rigor into my life: when I make predictions, instead of casting them abstractly into the air, I’ll post them here: einfall.slinkset.com. And I won’t delete my predictions — if they turn out wrong, I’ll keep them there, as permanent reminders to learn from.

Through accountability, honesty. Through honesty, improvement.

Thanks to Trevor Blackwell for the inspiration, and John and Brett from Slinkset for the List Hosting.

Notes: a friend of mine noted that most of my predictions seem ‘pessimistic’, in the sense that they take the form of ‘X will not Y.’ I would have to agree with him. But this is largely a byproduct of how these predictions were made – they’ve come from studying some field, working in it for a while, and coming to the creeping realization that one or more of the current approaches were doomed. Besides, much of the skill of experts comes from the ability to ignore false trails.

Further Reading: An excellent site for major predictions (often with significant wagers) is Long Bets.

This may sound weird, but I pretty much choose employment based on the promise of quality work. Other factors fade into irrelevance.

When I say quality of work, I mean more than the work environment, more than the magnitude of technical challenges, and more than the IQ of those I’d be working with. I want the opportunity to walk paths with the greatest hope of leading to first-class work. Nobel-prize winning kind of work.¹ This force guides me, and so inevitably I tend upstream of technological change. Money and prestige are mere proxies for what I really want: to develop and inspire fundamental changes in the way people live. That doesn’t mean I need tackle the greatest problems humanity now faces (yet). What matters is that I, personally, have a reasonable approach. So I must always remind myself to, as Richard Hamming says, ‘plant the little acorns from which the mighty oak trees grow’ — because small projects can, swiftly and strikingly, grow momentum and value.¹

I find little value in submitting myself to some company culture. I instead mean to develop my professional values, ambitions, and goals: for example, I would like to develop new methods, make them available by open sourcing them and make them popular by evangelizing them. I’d love to be given the chance to teach what I’ve learned. Excellent people bring ideas and perspective to a communities of makers. Given time, I think this will evolve naturally into a company culture worth having.

Most big companies grow faster than they could build trust, to a size greater than strong values can be supported. Natural culture is the product of alignment of creative philosophies, and in BigCo, this is too often replaced with virtual company nationalism. Fascism even. I find this is more than distasteful. I haven’t really learned how to work within it at all. I could devote my efforts to such an organization only were there deeply meaningful work to be done. And why bother?

Angling to be upstream of technological change, I bait unusual questions and find surprising answers. Give me the choice between a VP position at a big five media company with oodles of benefits, and, say, work at an early netscape or google for a totally minimal salary, and I’ll choose the latter every time. I’m pulled towards organizations where I can learn about organizing, rather than learning about institutional tradition. It’s not important for me to learn about how to run a large organization: if ever I do, I won’t follow of the paths of current captains of industry. Instead, I intend to help grow large, leaderless, open organizations, and so I’d do almost anything for a chance to work with Caterina Fake, or Linus Torvalds.

I want to work on something I find deep personal meaning in. I strongly believe in supporting open culture. I don’t think I’d work for long in games or entertainment unless it could influence some social change. I worked at MochiMedia because it made possible an income stream for small independent developers where none existed before. This finally opened up professional game development from BigCos. Now, much innovation in gaming emerges from bedroom studios. Independent game developers can now commit to their art in a way they before could not.

Similarly, I’d work at YouTube rather than Hulu, even though one’s a startup and the other isn’t, because they’re more interested in involving everyone in the process. As Clay Shirky says, they’re interested in ‘finding the mouse’.

I want to work somewhere where I can truly make a difference. Why am I working in technology at all? Archimedes once said, ‘If you give me a lever and a place to stand, I can move the world.’ Technology is my lever. I need only find place to stand. This makes me wary of startups that try to do good, but aren’t particularly focused on doing it efficiently. I wouldn’t work for most charities. There’s too little pressure on them to focus — the tempering influence of market competition is replaced by government demands for ‘accountability’, which arn’t nearly so powerful.

There are numerous ‘ecogreen’ websites out there that try to promote simple, green ways of living. These may be virtuous, however, in terms of minimizing environmental impact I think they’re somewhat irrelevant.² Saving plastic bags won’t lift a toe on our carbon footprint unless we find ways to either cut down on air and automobile travel, or do it more efficiently. And on carbon footprints — global warming is, I think, a red herring — there are thousands of nasty effects of pollution from, say, coal-fired power plants that will hit even if global warming doesn’t occur (though I think, probably, it will). Too much of China now wears breathing masks.³

I can see myself dedicating myself to the right company, so long as our goal, philosophies, and ambitions align. Yet these are stringent requirements. So far, then, I’ve found it necessary to reserve some energy and time for my own projects. So I must be open with companies: with most, I want only consulting work, to help them with some particular project, idea or problem. And I want to be completely, totally honest with everyone about it, because so far, the high road has never let me down.

Sincerely,
Danielle

———

Notes:

[1] – From the classic talk by Richard Hamming, You and Your Research. I don’t, particularly, apologize for my ambition here. Why shouldn’t I try to do first class work? The Nobel prize winning part is purely incidental. But this is the kind of work I mean — a significant contribution, one that people can build upon.

[2] – The free, online book Sustainability – Without the Hot Air is an excellent read. It is the first thing I’d suggest to someone interested in seriously starting into environmental matters. I shouldn’t claim that small contributions to green living are completely irrelevant — each does have some small effect. Perhaps raising the issue of green living in our collective consciousness will have an effect greater at second order than I imagined. But so many of our behaviors are misplaced. Many people, for example, go out of their way to buy ‘sustainable’ products at Whole Foods, say, when in reality, longer vehicle trips do more damage than almost anything you could buy. Many things are sold in a way to make you feel good about buying them. They don’t have any real effect!

[3] – This is worded provocatively, but pollution in China is a growing catastrophe. See ‘As China Roars, Pollution Reaches Deadly Extremes’ and ‘Where Breathing is Deadly’.

———

Thanks to Alex Lang, Ma’ayan Bresler, Nick Pilon, Colin Percival, Michael Nielsen, and Joel Muzzerall for reading drafts of this, and Charles Beatty, for sparking it.

PS: Certain misconceptions have been raised. Some feel that this is one demand of an over privileged generation. I reply to this here. Additionally, I am not, in fact, abandoning my startup. But I do need money, and a visa, so I am looking into either employment or seed funding.

True to form, when a discussion appeared on Hacker News I rushed to comment, and this article erupted from that attempt. The current scientific understanding of our cosmic origins is a mystery to the public at large, but it was only after I noticed the bewilderment of my fellow hackers that I realized how poor a job we scientists have done in conveying the motivation behind our discoveries.

This article represents an attempt to replace that sense of bewilderment with that of wonder. I want more than to explain what cosmologists believe. I want give people a deep sense of why we believe it, of how we’ve come to our current understanding, and of why we care.

Look close, and it seems the universe is lopsided.

The cosmic microwave background (CMB) is like a snapshot of the early universe. It was once all hot plasma, gas so hot that the atoms inside it were broken up. Because it was hot, it emitted light. Because it was dense, it was opaque: the light emitted couldn’t just pass through, instead it had to bounce around. But once cool enough, the universe became transparent: all the light could now travel freely. It was as if the photographic shutter of the universe was lifted.

The light from this moment became the cosmic microwave background radiation. Because the universe seemed to have cooled at almost exactly the same time everywhere, the CMB is, unlike almost everything else in astronomy¹, a picture of the entire universe at almost exactly the same moment in time. It is the best picture we have of the structure of the early universe.

The universe appears to have expanded evenly since then. We know it’s expanding now. Light is like a wave. Since the speed of light is constant, an illuminated object moving towards us has its wave crests squish together, turning bluer, and an object moving away from us has the distance between crests expand, turning more red. This is called a red shift. Since he knew the colors of certain celestial objects, Edwin Hubble was able to observe that the further something is from us, the more red-shifted its light, and therefore the faster it is speeding away.

Since we know that the early universe was hot, dense and small, and we know now that it’s cooler, sparse, big, and expanding, we can reasonably deduce that, long ago, there was a Big Bang. The universe exploded.

Strikingly, the CMB is almost the same everywhere you look. There are minor fluctuations, but even they seem to have the same distribution everywhere. The CMB, our best picture of the early universe, is extraordinarily smooth. It is one of the smoothest things ever observed in nature. This might not seem like a mystery. You might imagine that anything expanding, hot and dense would look roughly the same in all directions. It needn’t. Nebulae are formed by exploding stars, and they aren’t particularly smooth. In fact, in nature, it would seem, more often than not, that explosions are messy.

In 1981, Alan Guth suggested what might be called a ‘recipe for a universe’: inflation theory. Until then, nobody had come up with any good ideas for why the universe was so smooth and even. It is as if God² had pressed the entire universe with a cosmic clothes iron.

Guth said, suppose you started with pretty much any initial universe. Suppose you also had an extremely strong, extremely smooth field of energy. If this field started dumping energy into the rest of the universe, it would also evenly expand space itself.³ The universe would undergo a period of exponential expansion — inflation — having the effect of flattening and smoothing the rest of the universe. Inflation is God’s clothing iron.

A flat, smooth universe isn’t the only thing that inflation predicted. For example, at small physical scales, quantum mechanical fluctuations persist. During inflation these fluctuations are blown up as well, and these would seed, almost entirely, the cosmological structure of the universe. We see these fluctuations in the CMB. According to inflation, they are tiny quantum fluctuations blown up to a cosmic scale. They are, quite literally, the ancestors of our galaxies.

It wasn’t just that there were fluctuations. Inflation theory predicted a very specific distribution and type⁴. When people finally had the technological capability to check, that’s just what they found. The universe appeared, at a cosmic scale, astonishingly consistent with this simple theory. Yet recently our observational capacities have improved. A CMB survey called WMAP has uncovered several surprising and unexplained features, not all of which fit well with the our previous inflation theories.

If you divided the sky in half by tracing the orbit of the earth around the sun⁵, and compared, in each half, the size of big fluctuations, those between 3 and 5 degrees wide, you would come to the conclusion that one side has fluctuations outweighing the other by an alarmingly large amount. One side of the universe is bumpier than the other. Moreover, the difference is larger than would be accounted for by randomness, at least 99 times out of 100.⁶

This asymmetry looks real. It has been checked against every known experimental error and background effect astrophysicists have been able to think of. And if it is real, our previous inflation theories, with one field of energy to inflate the early universe, won’t work. They can’t account for this anomaly.

The authors Erickcek, Kamionkowski, and Carroll don’t merely point out this problem. They posit a solution. They describe another inflation model, consistent with our new observations. They suggest the universe had not one, but many fields of universe inflating energy. There’s just one problem. At least one of these fields needs to be asymmetric.

Where could such an asymmetry come from? It is possible that we’ll never know. Cosmology offers us the hope of uncovering consistent, compelling stories of our origins. Thousands of independent observations fit neatly in cosmology’s book. But while we may discover a few lost pages from our first chapters, we may never know all reasons why our book was written in the first place.⁷

Nevertheless, the authors make an exciting point. Wherever the asymmetry in the inflation field came from, it must have existed before inflation. It must have existed before the big bang. We had once imagined that time before our explosive beginnings would forever remain a mystery. Yet hidden in the CMB are hints of times earlier still. In this wonderful piece of work, the authors carefully consider what anomalies in the CMB could mean. And in the process, they may have discovered a way to look farther into the past than ever before.

Notes:

[1] – Since light moves at a finite speed, when we see something far away, we’re seeing light emitted in the past. What we see of something a light-year away is (at least) one year old.

[2] – I mean ‘God’ here as in a figure of speech. Feel free to substitute ‘Mother Nature’, ‘Allah’, or the ‘Flying Spaghetti Monster’ while reading.

[3] – What does it mean, exactly, for energy to expand space itself? It’s roughly analogous to blowing up a balloon. We know that the gravity of the universe, just like the elastic outside walls of a balloon, pull its contents inward. In a balloon, air pressure pushes against that inward force of the walls. During cosmic inflation, the inflationary force pushes against gravity. There’s one important difference though. We don’t actually know what the inflationary force is. Air blows up our balloons, but we have few clues as to what blew up the universe.

[4] – The quantum fluctuations predicted by inflation follow a nearly-scale-invariant random Gaussian distribution. These fluctuations show up in the CMB, and for the most part follow these predictions pretty closely.

[5] – The line dividing the two halves of the sky here is called the ecliptic.

[6] – Formally, this statement is true at at least the 99% confidence level.

[7] – There are some questions forever beyond our grasp. Even if we knew from where the Big Bang had come, we could always probe further, and ask where that came from.

An article on one bright young man, Moshe, recently appeared on Hacker News. For a long time I’ve been meaning to write about the subject, and what was to be a simple comment morphed into this essay.

The story of educational acceleration is an old one. Curious, bright children learn and explore rapidly on their own, and interactively with their parents. The world is like a playground for the growing mind. The child takes in everything. Eventually, these children find themselves mired in school’s morass. There are new adventures: more kids, older kids, a new environment. Yet kept in one place, individual attention of parents replaced by lectures from often overtaxed and uninterested teachers, their minds are left to go fallow. While some of school is new, and quite enjoyable, boredom and obedience, for the curious child, is torturous, a fact which lucky children and mindful parents come to confront.

Alternatives appear: skipping grades, dropping out, home-schooling, gifted programs, science fairs, participating in the popularity game, sports, focusing on musical or athletic achievement, playing hookie, becoming jaded.

After entering junior high I pretty much stopped responding to the world at large. Life rapidly degenerated. I quickly dropped out, and luckily my parents didn’t make me go back. At that time both of my parents were very busy with work, and so homeschooling couldn’t work for long. We discovered that college was much cheaper than private school, which didn’t seem very good anyway. We argued my way in.

I was given a huge amount of freedom. Like Moshe’s parents, and like many others, mine urged me to slow down — advice I tried to take, though my eyes were always bigger than my appetite. I was amongst a culture that respected me for good reasons. I shared so much more than with my age peers. I was learning so much. A growth spurt hid my relative youth — soon nobody suspected I was so young. I truly enjoyed my time in college. Nothing brightens a teenager’s world more than responsibility, respect, and freedom.

There are dangers. Early fame is a minefield, for many reasons, in many fields. I think it’s hard for most people to live up to past successes, the ‘promise’ they had shown. But there are worse dangers. Two extremes haunt the gifted:

It is easy to feel that what one has achieved is in some sense artificial. That one has been built up out of nothing — that really, respect is in a large part undeserved. ‘Imposter syndrome’ is a common term for this feeling. People can feel themselves to profoundly wanting. Phony. But this is by no means exclusive to the academically accelerated. It follows early achievement wherever it can be found.

Alternatively, one can fall into a trap of picking activities on the basis of their impressiveness. One can miss out on more intellectually simulating, enjoyable, and valuable things in life, all in order to participate in a vapid achievement contest. One strives to be the youngest smartest fastest most daring most broadly educated highest iq’d phd’d nobel prized ivy leagued quantum genius prodigy in the world ever.

This happens way too often. And it affects all of us. One can feel so small next to the prominent. Long shadows can reach deep into other’s egos. Perhaps worse, all of us have felt a need to be great at times. It can become overpowering.

It’s a trait that is unfortunately encouraged by some parents who should know better. It’s certainly encouraged by society at large. The wiser guardians of bright kids sometimes say that raising them is like having to constantly pull on the brakes. They must. The world keeps pushing them.

One of the saddest things is that it’s all futile anyway. Inevitably, you grow up. Prodigies become ex-prodigies. This is lost on too many people.

Beyond questions of ego and achievement, there’s another problem with acceleration. Those who advance so quickly do so by letting delight pull them. One falls in love with the rhythm of important, challenging work, with the excitement, with ideas, and with the culture of the mind.

But you can’t live life like this forever. One must learn other lessons while growing up. At some point, we all wish for a pause button. We all need time to pull our identities together. We all need time to relax, and there is much that we need that cannot be provided by an overscheduled life. It is far too easy, with successes behind you, with fame around you, with drive and courage within you, and with a love of your work pulling you, to let that pause button go, and to refrain from asking for respite before hitting rock bottom. I know. It happened to me.

It felt like the world was ending. I felt like I was abandoning my love and my duty. I felt incredibly selfish. And I was exhausted.

It was not so bad, it turns out. My life rallied. It is difficult to explain how much better I feel. And surprisingly, taking a break, and taking time to reset myself personally, has even started to improve my work. Much of that many be a function of happiness. But part of this may help even the happiest of people. Pressure to do great things can rush life. It is so hard to pursue daring thoughts in a life too hurried.

So my advice to Moshe, and anyone who finds a glimmer of recognition in these words: stay curious, let your love of something pull you, but do not hastily give all of yourself and find there is nothing left to give. Don’t engage in the achievement rat race. Listen to praise from those you respect, but ignore it from those you don’t. Push back when society eggs you on. Focus on things that matter the most to you alone. Ignore fears of being an imposter — be authentic to yourself, that is enough. Keep your hands on the brakes. Enjoy your freedom of time and mind. Pursue daring thoughts when the urge strikes you. And when your heart begs pause of the world, listen.

Further Reading:

The letter arrived silently. I’m embarrassed to admit that my body read like a chapter on the stages of grief. Shock. My stomach churned as I turned inward. Admonishments ‘not to take rejection personally’ meekly confronted universal doubts. Egos struggled against a rethinking of everything. Hours of discussion lapsed. Plans of what to do were floated, accepted, rejected, forgotten. Night passed to sunrise before sleep. Denial. I woke up recalling a story of one rejection mailed out accidentally, to a startup later to succeed. Thoughts strayed from their success – all I could register was the possibility of a mix-up. Anger. I stewed. ‘It doesn’t matter what they think. I know where they’re coming from, I know what they must think of us. They’re wrong! And we don’t need them anyway.’

Finally, acceptance.

YCombinator has its own personality. Their opinions are their own. They have perspective and wisdom and the will to apply it. But they have no more dominion over truth than the rest of us, nor are they immune to the blindspots that all beings must endure, nor would they suggest otherwise. It’s taken much for me to fully understand this

For years I’ve contemplated PG’s word as philosophy. It has changed the way I think. As PG writes, heroes are those of whom you’d ask “what they’d do in the same situation,” as for years I’ve asked myself. He is a hero.

But PG isn’t my conception of him, no matter how close his words have touched me. YC isn’t what I know of it, no matter how right I’d judged it to fit. And we are not what any others, no matter how wise, can ever hope to judge us.

Once can’t come to truly know people by their writings alone. Yet in my imagination I felt I could. Upon rejection, it couldn’t feel like a group in Mountain View had overlooked our ideas in favor of ideas more compelling to them. Their judgment felt universal. It took hours to move past the looming question: ‘How are we deficient?

Our idea seemed nothing but wonderful. Yet after taking a few steps away, and looking back, there are perfectly valid reasons for skepticism. PG is even on record stating that building applications for people interested in local events is a ‘perennial tarpit’. This isn’t a YC idea, even if it was YC inspired.

It started with a question: could software help people connect in real life? Nothing we knew about was any good. Nobody sane would sift through lists of events online. Dating sites mostly sucked. Put people in the position of a judge, and romance cannot bloom. Chat rooms sucked. There are trolls and perverts everywhere. And the internet at large has no location. Rarely would you find someone nearby. Even more rarely could you expect to know someone by their online persona: my YC dilemma writ large. And we knew, or believed, or hoped. There must be a better way.

But how could they believe us? With neither users nor customers nor a finished product, we have no proof of what is perhaps best described as a social experiment. All we have is code, determination, and the belief in our hearts that we are on the right track.

That belief can only come from imagining our success as we do. But our imagination is not theirs. Without proof, others cannot take the same leaps of faith. It is so easy to take rejection as a challenge to ourselves. It is only a rejection of our rendition. We are unproven.

How were we deficient? Perhaps we don’t see the problems others have seen: traps laying in wait for us just beyond the bend. Or perhaps it is because our imagination, try though we might, could not be made to fit with our heroes, and we were so certain of our shared viewpoint that we were blind to this possibility.

Much attention is being paid to YCombinator clones of late, and there are murmurs of the movement forcast more springing up. When the first batch appeared, they pointed out that YC would eventually struggle with deal-flow. Plenty of good startups were being passed over, so why not fund those? Too little effort was invested in explaining why one might prefer them to YCombinator, dooming them to the appearance of funding YC rejects. Later attempts focussed on a different limitation: bright entrepreneurs must contend with, in addition to harrowing expense, culture shock, and commitment, an immigration system unwilling to receive those planning to work for themselves.

Seed funding is limited by a third resource. For proven teams one needn’t far sight to bet dollars on success. But to bet on the earliest of early startups, run by strangers, not friends, toiling without traction, with working, complete technology, takes more than guts. To back such a startup requires they imagine the success for themselves.

There is hope for new players in the global seed funding game. But none will succeed as YCombinator does, winning successes from strange ideas with unproven players, unless like YCombinator, they cultivate a personality of their own, and call upon their imagination to see as the bright minds applying dream.

On such matters, an appeal to a higher power is appropriate. My friends use a variety of divination techniques, such as flipping a coin, tarot, or peyote. I, however, found myself reading an infrequently referenced blog post by Paul Graham (an orphan of the collapse of infogami).

“…as happened with lofts, the features that initially repelled people, like rough concrete walls, have now become a badge of coolness. Weird names are now cool, if they’re the right kind of weird. Nothing could be less cool, at this point, than calling a startup “cool.com.” A company with a name like that could not have arisen organically. “Cool.com” smells of a media conglomerate trying to create a web spinoff.”

and

“My favorite recent startup name is probably Writely. It looks so natural that even though it isn’t a word, you feel it should be. Anyone thoughtful enough to come up with a name like that is probably going to have good software.”

Even ordinary people have an extraordinary ability to glark meaning for a word newly encountered. A word that feels natural enough to exist in speech (‘I’m feeling ever so writely’) is quite a goal to aim for. People are sure to remember that.

I threw together a ruby script to create domain names from some simple rules and then check whois. I multithreaded it for throughput. (Ruby threads are easily invoked but apparently the threading system is not so powerful.)

My friend Nick pointed out that our startup targets the young, urban, and restless. I tried the suffix ‘libre’, meaning freedom, and slapped letters from A to Z and roots meaning city in front. Didn’t quite work. At some point, I ended up with collibre.com (it’s still available if you want it), though my partner in crime Alex said it was too much like collaboration, or a library. Finally, he said, ‘what about the root -opolis’. We plugged it in.

Hopolis.com wasn’t taken. We liked it. (don’t go there yet. It is only parked.) This name has only grown on us since then.

Anyway, yesterday I overheard some business types arguing over names. Someone mentioned capital. I thought to myself “hot diggity, let’s impress ‘em and get funded.” Alas, they were merely business students plotting a revolution in granola distribution (direct to the customer! Take that middle man!) Still, my script came up with better names than they did. Who’d have known so many things ending in ‘anola’ would be untaken?

So tonight I was hacking on hopolis, and six hours in, I started to shift into filling out the YC application. Now it’s almost done; only the questions I want to mull on and really ace are left. The sun’s rising — what’s there to do? Procrastinate, obviously. What better way to procrastinate but to update my neglected blog and save hackers from naming catastrophe?

I decided to post it here, and below. (the link goes to the Google Code project).

# Scans DNS for cool names, put together with prefixes,
# affixes, and middles.
# Usage: ruby DNSScan.rb
#
# Edit the domain composing code as you will. This is
# perhaps best used to make up words, eg. writely.
# Do not pass untrusted data to this program, whois
# has buffer overflows, so this may be insecure.

pre = ['Fox', 'Web', 'Net', 'Thought', 'Read', 'Work', 'Surf', 'Browse']
mid = ['']
suf = ['Stream','Flow','Link','Line','Focus','Scope','Glide','Fly',
       'Flight','Path','Wing','Eye','First','View', 'Sight','Vision',
       'Hunt','Wind']

# A single whois scan
def scan(x)
  found = system('whois ' + x.to_s + ' 2>1 | grep -E \
          'No match for|connect: no route to host' - \
          1>/dev/null 2>/dev/null')
  puts (found ? 'Free: ' : 'Parked: ') + x
end

# constructs domains (Note: It would be great if ruby had
# an outer join, so I could do (pre*mid*suf).each{...} )

domains = []
pre.map{|i| mid.map{|j| suf.map{|k| domains << i+j+k + '.com' }}}

# Makes threads (avoids blocking)

thrNum = 40
thrArr = (0..thrNum).map{
       Thread.new{ scan(domains.pop) while domains.any?}
}

while (thrArr.any?{|ta| ta.status})
  Thread.pass
end

Note: There are web services like this. They’re great, but they don’t seem to work fast enough, and I personally would rather have the name construction rules in code.

This problem is subtle, and wording is important, so I’ve reproduced the statement from Terry’s blog (emphasis mine) [editor's note: Terry didn't intend for a particular subtlety, and so has reworded his main post. He says that it had an 'unexpectedly interesting subtlety in its formulation, but was not the puzzle I had actually intended to write'. He's posted the original here]:

There is an island upon which a tribe resides. The tribe consists of 1000 people, 100 of which are blue-eyed and 900 of which are brown-eyed. Yet, their religion forbids them to know their own eye color, or even to discuss the topic; thus, each resident can (and does) see the eye colors of all other residents, but has no way of discovering his or her own (there are no reflective surfaces). If a tribesperson does discover his or her own eye color, then their religion compels them to commit ritual suicide at noon the following day in the village square for all to witness. All the tribespeople are highly logical and highly devout, and they all know that each other is also highly logical and highly devout.

One day, a blue-eyed foreigner visits to the island and wins the complete trust of the tribe.

One evening, he addresses the entire tribe to thank them for their hospitality.

However, not knowing the customs, the foreigner makes the mistake of mentioning eye color in his address, remarking “how unusual it is to see another blue-eyed person like myself in this region of the world”.

What effect, if anything, does this faux pas have on the tribe?

I’ve italicized the statement hosting the subtlety. It is known to everyone that if any tribemember discovered his or her eye color, they would dutifully commit suicide at noon the following day. And yet. The statement cleverly leaves open whether every person knows that every other person is as logical, or as devout, or as committed to the duties of ritual sacrifice, as they are.

To jump the gun, the statement doesn’t explicitly state the logical nature and devoutness of all tribespeople as common knowledge.

Hopefully examples will deliver me from vagueness. Consider tribes with one, two, and three blue eyed people on islands Galileo, Hippasus, and Russell. 2,3,4

On Galileo, after the gaff, the sole blue eyed unfortunate notices that none of his people share his tint of iris, and reconciles himself to a midday doom.

On Hippasus, one day later, our fated blue-eyed duo, fully aware that the other would have extinguished were he or she the only blue eyed resident, are forced to draw the conclusion that they are blue eyed, and so they disappear.

But on Russell, confusion sets in. Three blue eyes islanders know the two others. They know that each would commit suicide if they discovered their eyecolor, and they know that on the previous two days, no such thing happened. It would seem that the group would meet a bitter end. As the reasoning usually goes, each of the three, knowing the other two, would infer that the others continued survival could only mean the existence of other blue eyes on the island — theirs!

But wait. Let’s examine this more closely: take the blue eyed islanders, let’s call them, err, Alice, Bert, and Cecil. Alice sees Cecil and Bert; she knows that they as well as the rest of the tribe, and knows they’d indeed follow protocol if either discovered their eye color. What she doesn’t know for certain is whether Bert knows that Cecil would kill himself over eye pigment (even though he in fact would). Therefore, she doesn’t know for certain that a tribe with two blue eyed members would homogenize itself on the second day, she only knows that this would be true were she one of the members. She therefore cannot deduce that she has blue eyes, and the tribe’s taboo keeps information from spreading any further. Our trio is saved. 5

What would happen if all islanders knew that all islanders followed the same rules (as is explained in the variant posted by xkcd)? Instead of confusion, Alice would be certain that Bert and Cecil wouldn’t have survived were they the only blue eyed people, and so she must be the last. They all would die on the third day. Indeed, any group of blue-eyed 6 islanders under the same conditions would all meet their fate the same way.

There are other variants, Anatoly brings up a very interesting metaproblem where proactive tribe’s members might decide to save the others through early self-sacrifice. The analysis of this, and other metaproblems, goes very deep. It will have to be the subject of another post.

In any case, I hope I’ve thoroughly explored the magic. Or at least spoiled it!

As Terry writes,

“The remarkable thing about the puzzle, to me, is how such a subtle and seemingly academic change in the knowledge environment eventually propagates to have a concrete and dramatic effect.”

Notes:

I’ve been meaning to write that article for ages, in fact I was urged to do so by my friends in politics, since models from game theory such as the Centipede Game, the Prisoner’s Dilemma, and Mutually Assured Destruction remain dominant teaching devices, and Nash equilibrium strategies continue to be prescribed and accepted as applicable by acolyte bureaucrats. Yet I’m writing this article, and not that one. The power of social news compels me.

Galileo was famously persecuted by the Church due to his advocacy of essentially logical methods and conclusions. (though you probably already know that)

The philosopher Hippasus was said to have been either expelled from the Pythagorians or drowned at sea for his proof that the square root of two was irrational.

While Bertrand Russell’s magnum opus, the Principia Mathematica, was doomed at inception due to the underexamined implications of self reference, in this case, letting meta-knowledge probe only so deep means our Russellian islanders are saved.

I’m aware that some may object: ‘but doesn’t “and they all know that each other is also highly logical and highly devout” imply common knowledge?’ I concede that this might be an interpretation in some English. Not mine; what kind of language distinguishes knowledge from meta-knowledge but not meta-knowledge from meta-meta-knowledge? Sounds illogical. Oh wait.

A common misconception is that the brown eyed people would die too. But that’s not quite right since none of them are sure that their eyes aren’t actually green, grey, or magenta.

It’s not given much press, but the the Halting Problem is intimately related to Gödel’s First Incompleteness Theorem. Indeed it produces it as a correllary. Historically, Gödel’s incompleteness results were proved by hacking arithmetic into a Turing complete system, and this is still how they’re explained today.

There’s a one-to-one bijection between computability of a function and provability of a statement. Hence, the short, and generally accessible proof that the Halting problem is not in general computable for an arbitrary input is also a proof of the ‘most important, surprising result in logic’, namely, that some results, which have may have a perfectly valid truth-value outside a system, cannot be proven within it. One only needs the notion of a computer to follow this line of thinking, which is, in essence, what Gödel did. But the Halting problem is much easier to grasp. I’ve had children understand it, though it does take some walking through!

The interesting thing about the Halting problem is that it’s unsolvable in full generality, independent of whatever special capabilities the system has available. To see this clearly, consider the proof.

Question: Does there exist a (halting) program H which, given any program P, figure out if it would halt, for any input I?

Assume there exists such a program H. Construct a program T as follows.

(Program P, Input I) => (Boolean Halts):
if H(P,I) is true run forever
otherwise halt

Now, call T on itself, with itself as an input. Our assumption presupposes that H always halts. If T would halt on input T, then T will run forever. And if T would run forever on input T, then T would halt. This is a contradiction, so no such program H would exist.

This becomes philosophically and theologically relevant because it doesn’t at all depend upon the form that H might take. Suppose one has funky laws of physics, or happens upon a ‘book of truths’. Or suppose one is some kind of omniscient god, able to just ‘know’ whether something halts or doesn’t. All of these can be wrapped up by saying they’re part of H, whatever H may be, and even if you assume that H¹ always exists and is well-defined for some set, you cannot also have H be well-defined for programs which include calls to H.

In short, an omniscient entity can’t exist. Not even a god. When speaking amongst nontechnical types this conclusion is usually met with some form of drama: praise or otherwise.

‘Omniscience’, as it happens, doesn’t survive close inspection.

Notes:

[1] Taking the existence of H as given, convention names it a ‘halting oracle’.

The central limit theorem states that if you have many small, independent, random variables, then their sum is distributed approximately as a bell curve. Strikingly, almost everything is made up of many small parts, and these parts don’t tend to influence each other very much.

So much of what can measure seems to fit a bell curve. This is why the normal distribution works. Because this assumption tends to work well, it is usually taken as a matter of course. Students are taught it, lecturers preach it, researchers apply it, and startlingly few stop to question it.

Suppose the variables are not small, or suppose they’re not independent. Suppose, under certain conditions, the value of one variable would seriously effect another. Suppose we’re talking about the buildup of snow on a mountain slope. Most of the time, snowflakes can gradually build, without significant effect. But once enough builds, you don’t find snowflakes resting calmly upon a drift. What you find is an avalanche.

The sum total of snowflake movement isn’t what we might expect. The snowflakes on the top used to be lightly packed by the new, gradually coming down. The snowflakes on the bottom used to just sit there. But they’re not just sitting there. They’re moving fast, and they’re moving down.

The central limit theorem doesn’t always hold! If you have a model where, most of the time, a change in one part doesn’t  effect another much, but some of the time it really does, then you can’t assume that your outcomes will follow a bell curve. Entirely different outcomes are possible.

Traditionally, when you have some measurement that’s far outside of normal, you call this an outlier. In much statistical analysis, these are ignored or thrown out (for example, even the most extraordinary measurements won’t effect a median). This is useful if you want to study the ordinary behavior of something. But sometimes, the information you gain from outliers is by far the most interesting. Some of the outliers are expected in normal distributions (bell curves). But some of them are outliers because the model doesn’t apply. Some outliers are avalanches.

We started from an iffy assumption. Not everything is made up of independent random variables. Parts effect each other. Sometimes it’s violent, like a chain reaction, or our avalanche. And sometimes, it’s magical. Life is an example of this. If the chemicals that made up our bodies didn’t bond so strongly to one another, our DNA would unwind, and you’d be a puddle.

Statisticians try to account for this. One example on the tips of our tongues lately: finance. Roughly speaking, the ‘beta’ used in Finance means the volatility of a stock with the linear correlations with the company’s existing portfolio factored out. Despite this, there seem to be ‘six-sigma events’ happening all the time, things which, according to the theories, and ‘ordinary’ date, really shouldn’t be happening at all. ¹ What’s going on?

When small effects just add up simply, you can model them by what’s called a linear model (so called because if you add up small things along an XY graph, you’ll get a line). This doesn’t always work, and in fact, most interesting phenomena are non-linear.

Models are limited. They break down. And one can’t really account for every possibility. Financial markets can collapse due to dustbowls, and furthered by widespread investor panic. The destructive power of armies during world wars can be dwarfed by exhaustion and a powerful flu, though normally flu is beaten with chicken soup.² And planets full of ten story tall reptiles can be wiped out by meteorites. No number of small rocks would matter — usually they just ping off them. Bet they didn’t see that coming.

It isn’t too long before one fears running directly into Godel’s Incompleteness Theorems: checking all assumptions isn’t merely inconvenient, for most interesting problems, it’s impossible.³

The next time you hear about your monthly six, or seven, or eight sigma event, keep this in mind. Outliers are where the model breaks down. They happen more often than standard models would expect, and they often point out problems in the understanding of the system. If they start happening all the time, start mistrusting your statisticians. And your wall street.⁴

One can show the recursiveness of the model verification problem. Suppose one says to you: drug x worked better than drug y, with a 95% confidence. You might reply: what’s the confidence of that confidence? Somewhere along the line, they’ll have to say 100, and then they need to prove consistency. One can show that there are theories with unknown external variables and relationships (as are dealt with in statistics) aren’t even formally recursively enumerable,⁵ but even if they were, no such theory could contain a statement of its own consistency.

Back on planet earth, is there anything we should be worried about? Some who talk about the climate claim that, since basically the temperature of the earth just goes up and down naturally, it’s not really anything to worry about. But outliers can mark where models break down. Where systems change. This is a chart of temperature over the past few centuries. You may notice, to the right, an outlier. Maybe it’s telling us something.⁶

Further Reading:

1 – The Psychology of Human Misjudgment – Charlie Munger

“Now let’s talk about efficient market theory, a wonderful economic doctrine that had a long vogue in spite of the experience of Berkshire Hathaway. In fact o­ne of the economists who won — he shared a Nobel Prize — and as he looked at Berkshire Hathaway year after year, which people would throw in his face as saying maybe the market isn’t quite as efficient as you think, he said, “Well, it’s a two-sigma event.” And then he said we were a three-sigma event. And then he said we were a four-sigma event. And he finally got up to six sigmas — better to add a sigma than change a theory, just because the evidence comes in differently. [Laughter] And, of course, when this share of a Nobel Prize went into money management himself, he sank like a stone.”

2 – Wikipedia on the Spanish Flu.

3 – For the curious, a quick way to show this is by adding a ‘randomness’ oracle to a turing machine, and showing that you still can’t solve the halting problem.

4 – Wikipedia on the 2007 subprime mortgage financial crisis.

5 – As wikipedia states, a recursively enumerable language is a formal language for which there exists a Turing machine (or other computable function) which will enumerate all valid strings of the language.

6 – (update) For further musings on uncertainty and randomness, a terrific book is Nassim Taleb’s The Black Swan. A related essay is available here. Also intriguing is the working paper “Extreme Sample Selection Bias: Conditions that Cause the Correlation Between Two Variables to Switch Signs” by Tim Groseclose

“Everyone always discusses algorithms and such on this forum. However, there is one trait that is perhaps equally important as knowledge of algorithms – good problem solving strategy. By that I mean not jumping into solutions too quickly, thinking through everything before writing any code and such. Can anyone share what they’ve figured out that helps them? I’ve noticed that I have several flaws when I solve problems in general. I am too quick to start writing code. Also, I don’t think through my algorithms thoroughly. Also, there are many other things. Does anyone have similar flaws that they’ve already passed. If you have, please share your strategies for getting better at problem solving in general. Anything would be helpful, since I have the TCHS tournament coming up and I want to go home with some sort of prize.”

I wrote back, on May 10th ’07:

One thing I want to start doing is to make estimates of the implementation time of parts of my solution, how difficult each will be to code correctly/debug, and how sure I am that my method will work in the first place.

I’ve gotten very good at making estimates of time and space constraints. Now they are second nature. But knowing how best to solve a problem practically is still another area which I need to improve.

For example, for the last hard problem on TopCoder (circa May 9th, sorry, you’ll have to register to see it). I broke the problem down. Six steps toward victory! Five of those steps were really quite simple, though a few bugs crept in. The sixth was brutal — I could solve the problem by implementing a complicated comparison function object which recursively called itself on sorted sets of children. It was really crazy. To get it all working properly took nearly eight hours after the match.

Each bug, if not isolated, tends to increase debugging time by something like the square of the number of interacting components: the rate at which the number of dependencies would grow. Perhaps it would be simpler to either independently verify each component (by having pre-written tested functions or methods), or to organize the program, inasmuch as possible, as a linear pipeline or other simple acyclic graph. In fact, one might take a few seconds just to sketch the program idea out — how strongly coupled is it? What can I be sure of? How much of a pain will it be to debug? Can I make it less so? If not, can I anticipate debugging requirements and build in information gathering tools?

Furthermore, I was lured by that fact that, conceptually, ‘most’ of the problem could be coded simply. This blinded me to the fact that the code that was doing all of the work was almost completely unsupported by the auxiliary framework I had envisioned, and that the whole thing was a tightly entangled mess. Maybe the other coders can handle it. But I think it’s mostly because they know ways to keep things from failing — keep bugs isolated and visible enough to squash.

What I really needed to do was find a data structure or recurrence that was simple enough to verify piecemeal. I cannot be trusted to write bugless code. In fact, I should guard against it by breaking things down. This has the added benefit of making more clear what I need to do. I would need to remember less. Increasing my effective working memory has an impressively super-linear effect on my problem solving ability, how fast I can learn things, and also how much fun I’m having. I’ve come to realize this only recently, while studying for PhD qualifiers using index cards, scrap paper, two levels of books, one rough, one ‘canonical’, a wiki system of notes and a flashcard engine we made for it.

The last thing — and I think this is really more rudimentary, but I’ve yet to learn my lesson, know exactly what your variables are. In all of my experience this is the worst problem I have. Again and again I find myself with a few minutes to spare before the end of the coding phase (for readers unfamiliar with TopCoder, to get any points you need to have a solution that is (almost) exactly correct, submitted before the 1:15hr deadline). There is some minor error which I’ve detected. The tests give one more in some cases, one less in others, maybe it fails on an edge case. The tinkerer in me asks ‘hey! what happens if I turn this knob? Cross that t? Increment that i?’. So, I try it. Does it ever work? No. Not one damned time.

It’s voodoo programming, sure, and I make fun of it when I see others do it, but there’s just something — as that clock is ticking down, for some reason it feels like the competition is not about making sense of problems but just trying loads of things really really fast. And when I’ve defined an ‘r’ or an ‘x’ or a ‘q’ as some kind nifty function, where I only know the ‘approximate’ relationship with what I’m doing (up to a least significant bit, or maybe it’s part of the state of my program’s implicit automata that I used to speed up coding), when I’ve done this and the program nearly works except for this minor case or that minor case, it takes me nearly forever to disentangle what something is from what it is supposed to be. Even though I feel like I’m so close, when the pressure is on I just can’t handle it. It’s just way too complicated to do in my head when I keep having to watch the clock.

——

One advantage of TopCoder is that you’ll be constantly stretching your abilities. You’ll be solving many more problems than you otherwise might, and because of the time pressure you’ll learn faster, and it will stick better. I came across apropos comments by Philip Greenspun

1. A person won’t become proficient at something until he or she has done it many times. In other words., if you want someone to be really good at building a software system, he or she will have to have built 10 or more systems of that type.
2. A person won’t retain proficiency at a task unless he or she has at one time learned to perform that task very rapidly. Learning research demonstrates that the skills of people who become accurate but not fast deteriorate much sooner than the skills of people who become both accurate and fast.

and

Whatever the training task, the pace must be ruthlessly brisk. The learner should be expected to build at the same pace as an experienced developer. The difference between the learner and the wizard is that you expect the learner to make a lot of mistakes. The system as built may be awkward or not handle error cases properly. That’s okay. Training research shows that if you get speed now you can get quality later. But if you don’t get speed you will never get quality in the long run. We practice this technique in 6.916, Software Engineering for Web Applications, our course at MIT. Each student builds five database-backed Web applications during the 13-week semester. The first few that they build, during the course of the problem sets, are not necessarily elegant or optimal, but by the end of the semester they’ve become remarkably proficient, especially when you consider that each student is taking three or four other classes.

This rings remarkably true. Periods where I’ve needed to learn a skill in a short period of time have inevitably resulted in me learning them well. TopCoder is a terrific example. So was the high school math I needed to catch up on when I found myself in the university math department at 12. Cramming seems to have gotten a bad rap. I learn best intensely.

It’s not overarching. One of the most striking counterexamples is my friend Joel. When I met him in first year, he had already read code complete, and was working on his own games. He painstakingly engineered every assignment to be bug free. Extensible. Readable. Coherent. Most of us rationalised such behavior as making sense only because later assignments built upon the previous.

Joel lost the floppy he kept his work on. Most of us would have been upset. He was unphased. He did all his engineering for practice, and was happy to be able to do it again.

On TopCoder, he isn’t always the fastest. But he is the only competitor I know with a perfect accuracy rating. I don’t believe this skill was developed under time pressure. But I do believe that it entailed long hours, and sincere commitment to build something well. That is quite rare.

——

It’s been a while since I’ve done any of these contests: things keep getting in the way. Making rent, and taking care of other basic necessities has impeded my time for fun. Hopefully my skill wouldn’t have deteriorated in the interim. I actually have hope that I’ve unlearned more of the bad habits than my good. :-)

Further Reading

Misof’s description of the solution to the “RussianCheckers” TopCoder problem highlights a crucial part of the problem solving process: understanding how to reliably, quickly, and accurately implement a particular solution. The problem statement gives all the information one needs to solve the problem, but the task of implementation is so hard that only two competitors manage. Petr, the winner of the match, has a solution that’s a paragon of readability.

One of the economists scratched his head, drew a sip of black coffee from his porcelain cup, and mumbled something about how a large proportion of the physics department of Harvard University was hired by a trading company, with the lure of riches beyond the pale of the meager imaginings of the physicists (“you mean I can afford a house?!”).

But what did they do? Why them? What did the trading company see in them? I offered, from my experience in TopCoder, that trading companies appear to be mostly interested in hiring people who were very smart, and that maybe physicists at Harvard fit the bill.

“Uhmmm, no, not quite. I think it was something along the lines of specific expertise… something about, quantum… quantum field theory, maybe?”

I closed my eyes. Quantum field theory — gads! They’re going to use that for economics? Renormalization, Feynman diagrams, antimatter. I was aghast, finance would certainly be fun this way, but how on earth would anyone use it?

As my incredulity faded, it dawned on me. “Aha!” I exclaimed, “I think I know what they do! In Quantum Field Theory each normal mode oscillation of the field is interpreted as a particle, and all of these oscillations, coupled together, form the field. But we can just as well interpret an economic ‘price function’ as a field, and interpret each individual market player as a linear oscillator! The mathematics is exactly the same, so we can use all of the methods of QFT! It’s brilliant!”

I asked my friend across the table: “You must tell me. Did it actually work?”. He shook his head. “I don’t know. All of the results would be lost to the company. Perhaps if the stock rose inordinately it might say something, but how or why it happened would always be a mystery.”

Dejectedly, I pondered this. For perhaps the ex-physicists are happy, with their six-figure salaries and shiny urban lifestyles. But if I was them, well, I doubt I’d be able to handle keeping my application secret. And so I discovered yet another reason why I can’t live in the real world. I’m an intellectual show-off.

Michael Nielsen writes in to point out that Jim Simons, of Chern-Simon’s theory, is in fact the founder of Renaissance Technologies, a prominent hedge fund.

Tonight it’s winter in Berkeley. 53 degrees and raining, and outdoors, warmed by a heat-lamp, sheltered by an awning. I draw spiced apple cider through my lips. Classical music plays. An earbudded minority vote silently with their ears. Old men watch hooded students roll down the hills towards Telegraph Ave, Berkeley’s epicenter of hippiedom. Moist, newspapers ink the hands of activists, busily plotting the victories in the years long struggle to ‘save the oaks’. A young man lids a drink and smiles at me. Separated by glass, headphones, and 12 feet, I smile back. We wave.

There’s something magical about this place.

I don’t know anyone here. To arrive I flew four thousand miles from my place of growth. This place isn’t home. Yet there are few places that attract me so strongly. Modern life has been made private. And in doing so, life’s become a little lonely.

Builders of great cities have long understood that life would, but for misfortune, consist of more than work and one’s home. The vibrancy, energy, and community grown in what are sometimes called ‘third places’ played part in much of the world’s social, political and intellectual revolutions. The roles that the Roman forae, French salons, and English learned societies played in scholarship has been tremendous, as has been the influence of American chautauquas, worker’s taverns, and artist’s ghettos in social and political spheres. These public, accessible, talkative, comfortable playful places are magnets for folks of many stripes. Creativity can thrive there. Unconstrained by work’s implied unity of purpose, and decoupled from the tight bonds around one’s family and home, third places give marginal people, ideas, and voices room to grow, people to hear them, perspectives to challenge them, and food to help keep the conversation going.

Yet in much of the world, third spaces are dying, or being replaced by poor substitutes. Nearly half the USA lives in a suburb. Some try to define suburbs with density requirements, or as satellites of more major cities. But that doesn’t seem to capture the idea: almost all of San Jose is a suburb. Los Angeles, despite its skyscrapers, feels desolate in the same way, albeit in a different intensity as Jacksonville, Florida. We’ve heard it all before suburbs have too much pavement, too much repetition, too many box stores, too many cars and vans and soccer moms and private schools. But nothing so much makes a suburb if not a lack of third places. It may be the defining characteristic. There’s nowhere to go. There’s nothing to do.

So people fight back.

They watch TV; sitcoms and reality TV, predominantly: short, canned glimpses into lives where people are living. The same people, born of similar neighborhoods decades later, would watch YouTube over Friends, prefer to text instead of talk, and paste pictures in MySpace instead of a scrapbook. They’d hangout in different places, most marginal, most online. Yet despite these differences, the core behavior remains the same. People reach out subconsciously and compulsively to the world. They have a social itch, and nothing in their known universe will let them can scratch it the old fashioned way. Times have changed. Things are just too far away. Instead, they substitute.

Centuries ago, Shakespeare’s melanchoic Jaques proclaimed:

“All the world’s a stage,
And all the men and women merely players;
They have their exits and their entrances;
And one man in his time plays many parts…”

If the world can be a stage, then the web is valued stagehand. Millions now live essentially online. Working means sitting behind a computer. Home means sitting behind a computer. Hanging out means sitting behind a computer. The web has made possible access to millions more places than urban planners had dreamed of. It deeply affects our culture. Yet, in many ways, it comes up short. The web doesn’t serve apple cider, nor can it provide heat lamps for the evening chill. Eyes I catch won’t smile back at me. I won’t be able to catch eyes at all. To how many does it occur that when reading the New York Times, or Reddit, or YouTube, hundreds of eyes are following the same pages they are? Does it occur to them that comments on popular postings represent the opinions of only an assertive minority? Does it occur to the millions of highschoolers using Wikipedia that they can change a page at will? That thousands of others have requested the same references for the same civil war paper as they? So much interactivity of the physical world has been lost in the transition to the web. How much can we get back?

——

PS: To all you aspiring entrepreneurs, ThirdPlace.com appears to be parked. The tagline?

ThirdPlace.com

The Leading Competition Site on the Net

——

Further reading:

rokhayakebe describes online places as cities in a comment on Paul Graham’s Cities and Ambition

“I think the biggest shift in deciding where to live is happening “online” rather than “offline”. I am an online nomad.

I never lived and never will live in MySpace. I do not like the MySpacian’s message (Hey let’s try to see who has more friends and hookups).

I sometimes spend time at Facebook. I lived there for a little while until I realized I am not so much into keeping in touch and I had no friends in the few hours I spent in college. When they open their borders, that’s when I found that I do not like the Facebookies message (You should throw more pies and send more kisses)

I vacationed at Twitter, but it is not really my cup of tea. I still don’t get their message (Life is a popularity contest).

So Where do I live? Well, I live mostly in HN. Although I sometimes get into arguments with the habitants, I have yet to find another city that beats the intelligence, vibe, energy and support I witness here. I take a daily ride to Techcrunch City and NYT, but I make sure I come back home to HN and mingle with the people who live here.”