A conversation, from the point of view of any one participant, has these three elements:  

- the source: the other participants, either as individuals or as some collective they form for the purpose of this conversation  

- the context: as simple as an explicit topic, as subtle as an implicit one, as complex as a network of human relationships  

- your role: sometimes you are yourself, sometimes you are acting in a defined capacity, sometimes you participate in a conversation as part of a collective (like an audience)  

All conversations have these same fundamental elements. At the moment, our electronic conversations are subdivided and segregated according to ultimately trivial subclasses (personal email, IM, mailing lists, newsletters, feeds, web sites themselves, innumerable other variations on alerts), and the useful tools for managing conversations are scattered across similarly (arbitrarily) segmented applications.  

The conversation tool I really want will be built on the structural commonality of conversations, instead of the disparities. I want to apply email rules to RSS feeds, feed mechanics to mailing lists, contact lists as view filters, identities as task organizers, Growl formats as cell-phone notification styles. I want my email correspondence with my mom to be at least as rich as my soccer "correspondence" with the MLSnet front page, and I want my conversation with MLSnet to be as malleable as my own iTunes song status.  

I want, I think, for all my conversations to be structurally equal. I want to be able to look at them organized by any of those elements: by source, by context or by my role, and by the unions and intersections thereof. I want to see conversations in context when there's context, and I want there to more often be far more context than a list. I want to be able to understand my side of my conversations in aggregate, and for the memberships of my conversations to be effortlessly expandable, and for my computers to remember things my head forgets, even when I can't remember whether I knew them to begin with.  

I want, of course, the whole internet redesigned with this desire at its core, but I also want approximations of this in the meantime. I want Apple Mail or GMail to stop acting like somebody invented "mailing lists" and "address books" yesterday morning. Or I want Shrook or BlogBridge to speak POP3 and IMAP as fluently as RSS and OPML. Or I want Safari to look like Apple solving a problem from their own principles rather than letting somebody else define the form of the answer, or Flock aspiring to be a social application rather than just a browser with a few extra menu commands. I want Agenda and Magellan back now that I finally have enough information to justify them.  

I want us to have the conversations we could have if we didn't spend so much time just trying to keep track of what morbidly little we've already managed to say.

The main human information goal is understanding. Or wisdom, depending on your precise taxonomy. But either way, searching is plainly a means, not an end, and the current common incarnation of Search, which involves arbitrarily flattening a content space into a set of independent and logically equivalent Pages and then filtering them based on the presence or absence of words in their text, not only isn't an end, but is barely even a means to a means. This form of two-dimensional, context-stripping, schema-oblivous, answer-better-already-exist-somewhere searching is properly the very last resort, and it's a grotesque testament to the poverty of our information spaces that at the moment our last resort is often our only resort.  

The first big improvement in searching is giving it schema awareness. I doubt the people behind IMDb spend much time thinking about themselves as technology visionaries, but IMDb Search is a wildly instructive model of what is not only possible but arguably almost inevitable if you know something about the structure of your data. IMDb presents both the search widgetry and the answers in the vocabulary of the data-schema of movies and the people who work on them, not in "keywords" and "pages", and understands intimately that in IMDb's information-space search exists almost exclusively for the purpose of finding an entry point at which to start browsing. You go to Google to "look for something", you go to IMDb to "look something up"; the former phrase implies difficulty and disappointment in its very phrasing, the latter the comfortable assumption of success.  

On the web at large, of course, there is no meaningful schema, and it's impossible to make any simplifying assumptions about the subject matter of your question before you ask it. It is more productive to search in IMDb than with Google not because IMDb's searching is better, but because its data is better. But this does not even fractionally exonerate Google, or anybody else who is currently trying to solve an information problem by defining it as a search problem. They're all data problems. Google has the hardest version of this problem, since they don't directly control the information-space they're searching, but they have more than enough power and credibility to lead a revolution if they can muster the vision and organization. And anybody building an "enterprise" search tool has no such excuse; the enterprise does control their information-space, at least out to the edges where it touches the public space, and every second that can be invested in improving the data will be at least as productive as an hour sunk into flatly searching it.  

So if I worked for a Searching company right now, I'd start madly redefining ourselves tomorrow. We are not a searching company, we are an information organization company. The last resort is necessary, but neither sufficient nor transformative. I'd pull the smartest people I had off of "search" and put them to work on tools for the other end of the information process, reaching to the humans who are creating it and giving them the power to communicate not just the words of what they know but the structure of it, and to the collective mass of people to help them communicate and recognize and refine their collective knowledge about the schemas of known and knowable things. This is why Google Base holds the future of Google, and why you should sell your Google stock right now if they keep treating it as mainly a way for someone to buy your unused exercise equipment from you using a credit card. It should be the world's de facto public forum for the negotiation of the schema of all human knowledge, and if it isn't, every other decision Google makes will be forced by whatever is.  

But well-structured data, though necessary, isn't sufficient either. The good news for "search" companies is that improving the data is itself just a means to an end. Ideal data only encodes what we already know. The problems of useful inference from known data are hugely harder and super-hugely more valuable than the current forms of searching, especially when you realize that the boundary between private and public data is an obstacle and an opportunity in both directions not a wall to hide behind or run away from. The real future of "search" is in providing humans with the tools to form questions that haven't already been answered, and assemble the possible pieces of the answer, from threads of reasoning that traverse all kinds of territories of partial knowledge, into some form that synthesizes ideas that have never before even been juxtaposed, and onto which humans can further apply human powers where machine powers really fail -- fail because the machines are machines, not where they fail because we didn't take the time to let them be more thoroughly themselves -- so that they in turn can help us be more completely and wisely human.

I think it is becoming painfully clear that the web suffers from at least two critical design flaws, one of structure and one of usability.  

The usability flaw is the omission of real tracking and monitoring from the original browsing model. The "visited" link-state is no substitute for true unread marks, and HTTP response headers are not adequate building blocks for functional monitoring.  

RSS, born out of various motivations, is turning most coherently into a retrofit tracking/monitoring overlay for the web. My conversation with a feed is qualitatively poorer in context (and potentially in content) than my conversation with a web site, but it's qualitatively more manageable. The feed tells me when there's something new, and what it is, and lets me keep track of my interaction with it.  

This dynamic should have been built into the model from the outset, because without it the whole system does not scale in use. Providing it as an overlay, and a whole parallel information channel, is idiotic in every theoretical sense, but its pragmatic virtue is that a separate system can be built with far fewer technical and social dependencies.  

And while RSS is still nowhere near social critical mass among human information consumers, it is approaching a viable critical mass as a geek technology, and we will thus be increasingly tempted to lapse into thinking of it as a goal in itself, rather than a means. Witness, most glaringly, the fact that so far nobody, not even the people writing RSS-aware web-browsers, has attempted to use RSS to solve the original problem, which is making sense of the contents of a web site, not from it. And witness, almost as gallingly, the fact that we're pretending to think there's a future to the network model in which every reader is individually polling every information source every few minutes.  

In content, the same time-wasting cycle is going to replay in RSS that played in HTML: the new channel is initially lauded by sheltered tech geeks for freeing the "real" content from all the crap that used to surround it, and then quickly retaken by the forces of reality, which understand the commercial point of "all the crap". So ads and context will creep into feeds, and before long the content of RSS items will start to reproduce the whole experience of the originating web site, and there will no longer be anything streamlined or usably universal about the content of a feed.  

In scalability, the whole thing is just going to implode, or become horrendously convoluted as people scramble to patch over the network problems with proxies and collective batching.  

Of course, if we're going to have to rebuild the whole web for structural reasons, rebuilding the tracking/notification overlay on the current web is a throwaway project, but that doesn't mean it isn't worth doing, and it certainly doesn't mean somebody won't try to do it.  

If I were working for Microsoft or Apple right now (or, in theory, on Mozilla, but I'm not sure this can be done without corporate-scale backing), I'd have my R&D people putting serious work into treating RSS not as a content channel but as a source of the necessary metadata to build monitoring and tracking directly into the browsing experience. Forget trying to "teach web users about feeds", they shouldn't have to learn or care. Build the monitoring/tracking stuff around and into the browser where the user already lives and reads.  

If I were working for an infrastructure company, like Google or Akamai or maybe IBM, I'd have my R&D people hard at work on standards proposals and proof-of-concept prototypes for a cogently bidirectional subscription/syndication protocol that sends messages from the consumer to the source only when the consumer's interest changes, and from the source to the consumer only when the information changes. Quantum-leap bonus points for subsuming old-style email/IM messaging and web-browsing itself into the same new protocol. These are all most essentially conversations.  

And in the meantime, if I were working on any kind of RSS/OPML-related application, I would take a day or two to stop and think about my goals, not in terms of today's syntaxes but in terms of the flow of information between human beings, and between machines as our facilitators. I'd want, and maybe this is just me, to be working on something that not only improves the lives of people using the imperfect tools it has to work with right now, but would improve the lives of people even more efficiently if the world in which it operates were itself improved. Sometimes a broken window demands plywood, but as a tool-maker I dream of making something you won't just throw away after this crisis passes.  
 

(Of course, the deeper and ultimately more costly of the web's two design flaws is the structural mistake, which is the original decision to base HTML on presentation structure, rather than content structure. This is a monumental tragedy, as it has resulted in humanity staging the largest information-organization effort in the history of the species, and ending up with something that is perversely and pathetically oblivious to how much more than screen-rendering engines and address resolvers our machines could have been for us. In building this first unsemantic web we may well have thrown away more human knowledge than we've captured, and now we're going to have to build the whole thing over again, more or less from scratch, against the literally planetary inertia of our short-sighted mistakes.)

Here is my favorite current example of a good bad internet business idea. My wife and I have a number of friends with whom we often get together for variously collaborative dinners in various permutations of availability. Nearly everyone has some kind of eating preference, and several of our friends have menu-changing physical or moral constraints like wheat allergies, lactose intolerance and levels of vegetarianism. Keeping track of all the personal variables and their interactions is hardly impossible, but neither is it trivial.  

DinnerPeace, then, would be a web-based eating-constraint resolution service. Each participant specifies their own food rules, the dinner's host selects the guests for a specific event, and the service automatically combines the guests' needs to produce a composite constraint profile for the evening's menu.  

Later versions might add invitations, recipe-database integration ("I've got these eight people, a lot of spinach, and a good sale on Alaskan salmon; give me some suggestions."), pre-event/potluck planning discussions, post-event notes and recipe-sharing, and historical context ("Oh, and I don't want to make anything I already made for any of these people before.").  

As a stand-alone business this is a fairly dismal idea. Users won't be willing to pay much, if anything, for what is at best an appealing minor convenience. Food-related advertisers might be interested, but the only ones really likely to benefit would be grocers local to the people preparing dishes, and there probably won't be a critical mass of participants in any given locality, soon or ever.  

More significantly, though, an independent service is simply the wrong model for the idea. It requires the builder to provide and maintain all the framework of a generic social service (notably member- and discussion-management), and requires the users to join another thing, keep track of another set of credentials, and of course provide a bunch of information whose privacy they have to consider, and which is unlikely to be reusable in any other parallel or future venue.  

The biggest jump-start for building little good ideas like this would be a pre-existing public infrastructure for distributed identity management, with portable authentication, ratification of trust, communication uniqueness (that is, your new managed identity is sufficient contact information for IM, email, etc.), arbitrarily extensible personal profiles, integrated personal past-and-future calendar-handling and straightforward control over what information is exposed to whom. This needs to be at least as easy and ubiquitous as email is already. In the new world, in fact, this (and not just email) needs to be the new baseline for online presence, in the same way that the baseline for telephone presence grew from home-phone to home-phone+answering-machine to home-phone+remote-messaging to cellular.  

This new baseline identity system would get DinnerPeace and everything like it (including much bigger things with ultimately the same structure) out of the commodity headaches of name arbitration, password resetting, access control, scheduling, elemental data-storage, history, recovery, etc., and leave each inventor to put all their work into their idea's unique characteristics, which in the case of DinnerPeace are really only a reference schema and vocabulary for the representation of a person's eating constraints, and the associated reconciliation logic for sets of these constraints and their histories.  

The business problem may be a little harder than the technical problem, but it is of the same shape. Along with the new identity system must come a distributed microcommerce system of which ad-click commissions are only the distant ancestor, their replacement being much closer to a pervasive method of tracking and apportioning credit for all the influences on each spending event, including the new possibility of tipping as a networked and optionally aggregated act.  

Thus DinnerPeace, and all the other little pieces of a smarter new world, mostly shouldn't need "business plans" in the old sense, nor investors or funding or stock or even companies. They shoud live or die or evolve based on their usefulness, and profit or not based on how much commerce they touch. DinnerPeace should generate one little trickle of money from how it affects what its users buy, another from its users' direct gratitude, and a third from its share of its users' collective appreciation of all such services they employ. This money flows in outgoing trickles, in turn, to the contributors to the service's logic.  

Probably the trickles from one idea usually don't add up to a living for even one person, let alone several, but then most of the little ideas from which they flow are not life's works, either. They are inspirations of moments, and the work of hours or days. The new world is improved by little touches, and rewards them with little gifts. And if it becomes less compelling to dream of retiring on windfalls of luck or greed, then maybe it will become easier to live by caring.

[Postscript to Content and Presentation Are 3...]  

Arguably the most fundamental specific semantic failing in the current HTML approach to information communication is that it does not effectively distinguish between a) content unique to the individual information node represented by the page URL, and b) navigation and context appearing on the same web page but belonging conceptually to a higher containing node. Thus, most obviously, indexers have no way to reliably attribute text-matches or links to the correct node. Less obviously, perhaps, this makes it difficult for reading software to easily recognize that two URLs present the same content in different contexts, which is currently the shape of quite a bit of content abuse, or different content in the same context, which at this point is probably the shape of most legitimate web information. The real solution still lies in separating content structure and presentation structure, but a microformat-style approach might help a lot in the interim.  

PPS: There is also currently no good method for separating the content and context components of a URL, either, meaning there is often no user-visible URI for the content node itself. This may be moot most of the time in the new world, since content will either have a context imposed upon it or will invoke its own default context. But standardizing URL formats in the short-term may be even harder than standardizing class-based pseudo-semantics...

At the moment, the tools for online publishing were mostly designed by software people for an audience of other software people, so it shouldn't be too surprising that by real (i.e., pre-/non- online) publishing standards most of them are not only fairly terrible in particulars, but basically comprehensively alien and unresponsive down to the conceptual model.  

The reason the HTML-tables-vs-CSS-blocks war wasn't over ten minutes after CSS was invented, for example, is that the CSS float/span/margin/escape model chose (more in ignorance than defiance, I'm sure) its own internal geometrical cleverness over the potential applicability of centuries of conventional information design. The most basic tool of scalable information design is and has always been the layout grid. CSS is capable of producing a simple layout grid, but not simply. Arguably it's even harder to recognize a page layout by reading its CSS than it is to write the CSS. Structural elements are structural by convention if you're lucky, coincidence if you aren't, and certainly not by nature, and are unavoidably cluttered by non-structural formatting elements.  

An HTML table isn't a good tool for page layout, either, but for the most part a simple layout grid can be expressed by a simple HTML table, and with decent code-indentation it is possible to more or less grasp the presentation structure of a simple table by looking at its code. The fact that tables require the content and the structure to be intertwined is a huge problem, and the fundamental misconception of the scheme emerges the moment any kind of table-nesting is required for page-structural and/or content-structural reasons. The non-programmer's mental model of a box is an actual box. You can put small actual boxes inside larger ones. If you forget to put a bottom on a small box inside a large box, you get a bottomless small box inside a normal large box, not a deformed small box falling out of the bottom of a mangled large box. Nobody, not even a programmer, intuitively thinks of object construction procedurally, and nobody but a programmer (and not most of those) would want to.  

Ironically, painfully, the closest thing in current web technology to simple modeling of simple presentation structure is technically the most flawed: frames. I would recommend their use for almost no public purpose, and frameset code is no pleasure to look at, either, but at least frame definition is separated from content (too separated, in fact, but separation is still a good idea), and its grammar operates by explicitly specifying structure from the outside in (with recursion, albeit awkwardly), rather than calculating it by implication from the inside out. If frames had been defined as page (and sub-page) elements pre-declaring layout structure, rather than window elements that contain content pages by reference, we might have had something.  

While we're redoing the layout model, though, we also need to fix our basic misunderstanding of the natural structure of repeatable content presentation. Any remotely enlightened programmer "knows" that content should be separated from presentation, but these are actually three things, not two: content, presentation structure, and actual formatting, and in practice the formatting can usually be further separated into content formatting (the formatting of the information unique to this content node) and page layout (the site-identity and navigation and other framing stuff displayed around it but conceptually invoked from context).  

Take this furialog entry itself, for example. Its content structure includes a title, a display date, a publication timestamp, a set of tags and a content block. This content structure is then mapped into a presentation structure for this web page which is quite a bit simpler: just a header and a body. The title, display date and tags are concatenated into a single formatted block for presentation purposes, and the timestamp is not (in this format) presented at all. The formatting rules, then, are applied to the presentation structure, not the content structure. This is important, because elsewhere on this site content with very different content structures gets mapped into the same presentation structure, subject to the same formatting rules, and thus it is possible for me to define a single rule-set that drives the middle layers of the production of all my content, a single page-layout that drives the outer layers, and appropriate extensions to which the sub-formatting of special-purpose inner blocks like tables and photosets can be delegated.  

The usefulness of RSS, similarly, is largely due to the fact that it defines a common presentation structure into which dissimilar content structures may be mapped. Different feed sources may put semantically different and/or compound information into the title and description fields, but the writing software doesn't have to know or care about the formatting rules, and the reading software doesn't have to know or care about the content structure. The drawback of using presentation structure as an interchange medium, of course, is that the reading software can't get the original content structure even if it could do something interesting with it. Human readers can re-interpolate it, but for the machines RSS tends to be lossy.  

This was also, maybe obviously, the original design concept for HTML: standardize a presentation structure that can then act as universal intermediary between content and screen. RSS gets away with its limitations because it's an alternate channel, but as the web emerged, HTML was the web, and universal intermediation quickly proved unsatisfying in both directions: writers couldn't exercise enough control over formatting when they needed or wanted to (and programmers often assume "want" is always the right word here, but publishers understand that publishing is different than just writing), and machines couldn't add enough processing insight without finer-grained and/or domain-specific data schema.  
 

Here, then, are some of the elements of the new model I want, some of what, from twenty years of working with information in both publishing and software spheres (and this isn't what I thought after ten, so maybe it won't be what I think after forty, but is that because I change, or because the world does?), I believe the whole system needs in order to fulfill its potential as a medium at once for direct human communication and human understanding supplemented by computer reasoning:  

- Not only should all meaningful content begin in its own schema, but this machine-parsable content structure should, inherently and automatically, underlie and accompany the human-visible presentation and formatting of all information. Imagine, if this were our only problem, adding an Elemental XML section to every HTML page, so that each page would carry its pre-presentation data in addition to all of its other markup, and splitting the View Source command into View Presentation Code and View Source Data to offer different tools for each. (Microformats are a similarly motivated effort, establishing presentation-structure conventions for particular kinds of content structure, but conflating the two kinds of structure is only reasonable as a short-term tactical compromise in a system where there's no good way to separate them, anyway.)  

- The mapping of content structure into presentation structure must be so trivially easy, and such an obvious entry into a richly distributed universe of sophisticated standard lexicons and templates, that nobody will even be tempted to try to skip the step by pretending that their content doesn't have any other structure than its presentation. Imagine, as you're publishing something, that you could chose dynamically from not only your own repertoire of formats and the templates provided by your authoring software, but from a social universe of templates, of all granularities, available from anyone who has one to offer. The laborious specific control of your own formatting, or the flexible custom extension of your presentation structure, should be optional opportunities for the motivated, not required burdens of participation.  

- The layout model should be based on recursive containment from the outside in (probably with layers, for graphic depth). Exceptional behavior, like violating margins, must be declared explicitly, not arise from routine errors in normal use. It should be trivial to map one element of presentation structure into one formatting container, to flow multiple elements of presentation structure into one formatting container, or to flow one or more elements of presentation structure through an arbitrarily linked series of formatting containers. We've learned a lot about UI and usability since Quark and PageMaker were invented, and not everything we ever wanted to do in books and magazines applies the same way on a screen or in an interactive environment, but in many ways the screen world is only now approaching the point where print was before computers, so as is already beginning to sink in about typography, more old lessons apply than not.  

- Where the new medium has new abilities, our tools should be granting new powers to us, not demanding new chores. AJAX is a clever retrofit of what should have been a native idiom. Forget paging vs scrolling, Google Maps is how everything offscreen should work, the computers dealing with computer constraints like bandwidth and memory without bringing people into issues that don't concern people. And if content were encapsulated intelligently, it wouldn't take Flash or iframes or scripts to make a piece of a page do things a whole page can already do. The web is a bad application environment kludged onto a bad publishing environment, and it ought to be a composite environment in which publishers realize that sometimes they are publishing applications and experiences and devices, not just pictures and words.  

- The separation of content, presentation structure and formatting is necessary for everyone's purposes, not just the auteurs and the programmers and the machines. Good, reusable, maintainable presentation design, even when it's done by dedicated designers, should always work first from presentation structure to formatting rules. The handcrafting of individual exceptions should be an elaboration on a framework of rules, not a repetitive substitute for making the effort to understand patterns. I see way too much CSS code that specifies individual pixel-unit fonts and margins and paddings for every single id-numbered div in an entire page (or, worse, for every id-numbered div in a whole set of pages, so that the style-sheet can be "reused" across all of them). Sometimes this is a result of having misused the (X)HTML to model the content structure, rather than the presentation structure, so that the formats and the selectors don't line up right, but I bet it's more often just laziness. It's always faster to add a pixel than to generalize a rule about why, but it takes an incredibly tiny number of exceptions to complicate a rule-set so irretrievably that it can only be modified by further exceptions. For all but the rarest purposes where chaos is the order, the fewer rules the better, and the square of the fewer exceptions the better.  
 

But fine. No matter what you credit as partial precedence, ubiquitous electronic publishing and communication as social infrastructure are at best in their adolescence, and it's amazing we've gotten this far with these primitive tools. The willingnesses to restart and rebuild are in our character, just as surely as myopia and impatience are our curses. Generations are how we learn.  

So yes, this means starting over, but the new ways, when they're right, are easier than the old ones, and we only shrink from them out of fear of change. In place of the current HTML/CSS information foundation we will have data (content structure), mapping (content structure to presentation structure), page layout (the containers through which the presentation structures flow) and formatting (typography, color, spacing, ornamentation and illustration within those containers). Any of these can be implied from defaults (set by the reader, writer or both), included by reference to external sources, included by recursion, included conditionally by context, or defined inline. Most of this will still be mediated by tools for most people, but the tools will be simpler and more powerful, and will let us focus more on what we're saying to each other, and less on how.  

And since the new way delivers both the formatting and the semantics, this will be a practical revolution that actually subsumes a theoretical one, better human communication interleaved with better machine communication, and thus the foundation for both better shopping and the dream of the internet being something qualitatively more profound and transformative than shopping.

My online information life is easily complicated enough for me to be sure that our current tools and models for how a person relates to information connectivity are still wildly immature. "Web 2.0" is a hopelessly inadequate vision of the future because the separation of "Web" as its own world is exactly part of the problem. My email, chat, feed-reading and web-browsing software perform intently interrelated (to me) communication and monitoring functions with a magnificent (to them) disregard for each other, and often for me. Only slightly less glaringly, but of increasing significance, my information publishing activities (both conventional and micro-) go on in a similarly near-complete isolation from my information consumption.  

What I need, and what everyone will need before the net can be considered an inclusive participatory extension of human social communication, is a much simpler, more straightforward and more agile mapping of system tools onto the four fundamental components of information interaction:  

- My information.  

- Distribution of my information to others.  

- Monitoring the flow of other people's information.  

- Consumption of other people's information (including connecting it back to my information).  

At the moment, this whole cycle is a mess.  
 

Information  

At the moment there are marginally acceptable tools only for the most tightly constrained special-case forms of information creation. I think person-to-person email is now mature enough, for example, that the nature of the tool set isn't excluding anybody who would otherwise be eligible, although obviously economics, technology and literacy still post significant obstacles to many people getting to use those tools in the first place. As soon as you try to raise the level of function and abstraction above the individual message to or from a known correspondent, though, the tools begin to show their limitations. Communicating with groups is cumbersome. Communicating with strangers is almost systemically incapacitating. Carrying on extended or episodic conversations is difficult, and relating the accumulating bodies of correspondence to the personal relationships they nominally express and inform is so poorly supported by the tools that I suspect most users effectively do not retain any value from their electronic interpersonal correspondence outside of their own heads.  

Nothing beyond email is even remotely comparable in developmental maturity. There are individually decent tools for extremely rudimentary self-publishing in the forms of simple streams of text and photos, and anything beyond that falls off into the usable domain of a vanishingly tiny minority of participants.  

Of course, arguably the only mature pre-connectivity computer function was ever unstructured word-processing, anyway, and if creating and managing structured information for their own purposes is beyond most users, then it hardly matters that there are no easy publishing methods awaiting that information. And thus any new concepts of syndicating a user's own information out to external forums, or re-consolidating distributed contributions back into central management and retention, don't even have a foundation on which to build.  

Thus it seems to me that the first thing we really need, underneath all of these tools and before we really start talking about communication at all, is an underlying data system, as opposed to (but just as native and optimized and standardized as) a file system. All our information creation tools should be manufacturing data, not files, and always with the bias towards representing that data in the most application-neutral, self-describing, reusable and standardized way (like Elemental XML, for example, or some isomorph). It should not only be effortless to exchange information between, for a crashingly trivial example, iTunes and Excel and your blog sidebar, but more than that, the way everyone (people and systems alike) should be thinking about the process is that the data exists independent of all the applications that merely happen to manipulate it and give it back.  
 

Distribution  

Possibly this is just a function of my own egotism and information-retentitive nature, but I continue to think that my information system should remember my information first, and send it elsewhere second. Moreover, I think that ultimately we will understand that email is merely a historically earlier-understood instance of the same general distribution problem as blog publishing, photo sharing, restaurant-review syndication, collaborative filtering and everything else. Put another way, general information publishing/sharing tools must mature to the point that they accomplish the personal-correspondence special-case as easily as (and preferably more easily than) our current single-purpose email tools.  

The large conceptual shift that needs to take place in the rest of the information world, as the new storage model makes it possible, is away from the assumption that a sharing format is necessarily an authoring environment. Collaborative tagging, for example, would be much more effective if annotation was a native function whose results could also be shared. Sharing (including full privacy control) should be an elemental function of the underlying data system, so that (for example) you don't "export" or "upload" a photograph to Flickr and then assign it tags and descriptions and sets, you tag and describe and group photos for your own purposes, and then choose some of them to share via a particular online medium.  
 

Monitoring  

Although several different ideas commingle in the current state of RSS/Atom feeds, the two most-central innovations are the provision of an automatic monitoring framework for the otherwise manually-browsed web, and the creation of a parallel lowest-common-denominator content format to go with the style- and context-heavily publishing forms used on the web.  

These will need to be disentangled, because the monitoring function properly belongs to a higher level. A conversation should not be constrained to different monitoring tools because it happens to take place in email, or on a mailing list, or in a group real-time conference or in the comment thread of a blog. Just as all the forms of content sharing should arise from a common creation and storage framework, we need a general form of monitoring that subsumes the current functions (and far exceeds the current usability) of email inboxes and filtering, IM buddy lists and presence, on-screen bezel/pop-up notification, RSS updates, menubar/Dashboard/system-tray widgets, SMS alerts and even web-browsing history and general read/unread flagging. For the new monitoring system we must figure out significantly better ways of understanding a user's dynamic segmentation of their monitoring needs along the continuum between urgent active notification and ongoing passive tracking. Ultimately I want a single console to watch, or more precisely a single logical console that can take multiple particular forms tailored to my different mental and physical modes, and adjust its common reporting to the subtly and radically differing natures of particular information sources.  
 

Consumption  

The corresponding evolution in information consumption, as is almost implicit in the other parts of the system, is that consumption is not really separate at all. What you read can be as much a part of your information flow as what you write, and the nature of your relationship to what you read should not change based on the incidental mechanics of the medium. The same human conversation could take place on IM, in email or on a web site; the same options for retaining and correlating and re-using should be available in all those cases. At the moment the tools for bookmarking web pages are only narrowly adequate, and the tools for usably retaining web information are nearly non-existent. We email ourselves web pages; this should prove that there's something very important missing.  

And, too, as we are only barely starting to understand with tagging and blogging, what I read flows into what I write, and into all kinds of information that I create implicitly and may or may not want to use and share. Conversations and connections flow through all this information, or try to. The new information world will understand and encourage and benefit from this flow.  

The new information world will be formed of this flow.

It is becoming increasingly possible for separate systems to perform each of the six major functions of data applications: storage, transformation (including creation), categorization (including tagging, indexing, search and retrieval), visualization, monitoring and the administration of trust.  

Historically, of course, these functions were usually not only performed by a single unified system, but mostly limited to that system. At its most insular, old-world applications entailed embedded storage in proprietary formats, integrated authoring tools and UI, integrated (if any) notification, and integrated (if any) user-management.  

In the old world of personal data applications, like spreadsheets and word-processors and whatever, standardized file systems at least separated storage and categorization from application logic (you could put your Excel file on a floppy disk, or in your "Budgets" folder). Semi-standardization of formats helped open data transformation and/or visualization a little bit (you could use Word's search-and-replace tools on an RTF file, or run Crystal Reports on your Paradox databases), but published formats are not quite the same thing as open formats. And monitoring and trust were usually expendable for personal applications, or solvable a function at a time.  

Old-world online applications changed the distribution of insularities. You could actually use several different tools to send, receive and monitor CompuServe data. Prodigy let you use any tool you wanted, as long as it was a construction-paper hammer designed by runt warthogs for use by cartoon infants. But the online service very clearly owned the physical storage, the content space and the identity space.  

The early web was a combination of progress and regress, pretty much no matter which directions you think those go. HTML offered the tantalizing prospect of separating the presentation logic from the data structures, but in practice browser convergence quickly resulted in this being true in only a somewhat obscure development-tools sense. You could produce your HTML files with different software than they would be read with, but you still had to take the client constraints heavily into account. HTML files could be moved around fairly easily, but cross-server transclusion got pretty ugly the moment you tried to move much beyond linking. And identity management was reinvented from scratch anywhere it was unavoidable.  

But we now have at least nominal rudimentary pieces of the ability to separate all of these. XML offers an interchangeable application-neutral storage format (or at least a meta-format), XML+HTTP gives us a way to virtualize storage (as long as we don't virtualize it too far), Google has demonstrated the scalable separation of categorization and some amount of visualization, and RSS is at least a conceptual step towards the separation of tracking. LDAP separates identity management for at least certain kinds of communities. These may not be the solutions, but they are indications that the solutions are possible and closer.  

But the next steps, in all cases, are huge, and at least as difficult culturally as they will be technically.  

Storage  

All systems must be prepared to handle external storage transparently to the data's owner, whether this actually means live reading and writing over the network or caching and mirroring to simulate it. An indexer must be able to hand you back the indexes it makes and updates, an image organizer must allow you to store the images on your own server, etc.  

Transformation  

All data must be stored in as neutral and open a format as possible. Application-neutral information must be tagged in standard self-describing ways. Proprietary information is acceptable only when mandated by definition (for internal security functions and precious little else), and where necessary must be clearly identified and attributed. These will be practical imperatives, not just moral ones. Secrecy is fragile, and the net routes around it instinctively.  

Categorization  

Anything that exists can be categorized. In many cases, the categorization will end up being qualitatively more valuable than the original information. The only difference between data and meta-data is that meta-data is data the owner of the thing didn't anticipate or provide for. The more fluidly a system can re-integrate the meta-data it spawns, the more powerful it will be. The more afraid you are of your audience, the faster they will depart.  

Visualization  

Similarly, the more readily a system opens itself to external visualization, the better off it will be. Whatever it is you own and control, it's never more than part of the experience. The default techno-social goal of a data application is to be the reference source for some kind of data. (The default business goal is to have some way to make money, not from that data but from that status.)  

Monitoring  

Various malformed and over-constrained attempts have been made to generalize the problems of monitoring, change tracking and notification into email, IM, RSS, Trackback, Konfabulator, Dashboard and countless proprietary and special-purpose schemes. The next generation has to supply a version that scales to the entire world, including not only its size and its bandwidth but also its heterogeneity and its self-organization. The new system has to rationalize all flows, including the malevolent ones.  

Trust  

Ultimately, though, the native currency of the new connected world will be trust. Every interaction of people and systems relies on it, usually inherently and implicitly. Existing systems have mostly survived on trust by exclusivity (firewalls, closed networks, internal identity management) obscurity (mostly self-selection) or informal accountability (feedback and self-policing). None of these scale. The new identity systems must be built not to administer specific applications but to provide universal credentials that verify a user's membership in defined communities. The new data systems must be built so that unknown individuals can be accepted on the basis of delegated authority. In the old world people were "users", users existed inside careful boundaries, and outside of those boundaries all there were were names. In the new world, people are the signals themselves, and a name is a name only by virtue of some authority, and maybe that authority by virtue of another one. In the new data world, where the scope of the network is as big as the scope of the planet, and the size is exponentially larger, the primary component of every transaction of storage, transformation, categorization, visualization or monitoring will be the intimate initialization of the basis of trust under which any two of us say anything to each other at all.

An abacus is a state machine. It executes no instructions, and maintains no history, but it does store a single state, semi-persistently and nearly-infinitely rewritably, and it stores it in a representation that facilitates operator-initiated state-changes of certain types. An electric typewriter with a single-character backspace function is approximately equivalent in computational terms. Both of these are very useful devices.  

A typewriter with a multiple-character backspace function has both state and memory. The simplest electric calculator has both state and automated instruction execution. A semi-modern calculator has state, instruction and memory, and at this point we can call it a basic computer. The subsequent history of human-computer interaction design has been a slow process of iteratively transcending decreasingly unimaginative understandings of the implications of state, instruction and memory.  

The conceptual breakthroughs of the earliest text-processing programs were 1) that semantically non-numeric information could be represented in numeric memory, 2) that semantically non-mathematical operations could be modeled in mathematical instructions, and 3) that quantitative increases in memory capacity could enable qualitatively different uses of that memory. Further thought about representation led to storing formatting in addition to text itself. Further thought about instructions led to the automation of layout operations, and the addition of text-processing operations like search-and-replace. This makes for a more interesting state-machine than an abacus, but still effectively a state machine in most user-apparent aspects.  

The conceptual shift from state machine to information appliance can be reduced, symbolically, to model-altering insights embodied in three perhaps seemingly incremental features. From the critical realization that the computer's representation of information could include more than the ostensible current state came the radical notion of Undo, and later its extrapolation to Undelete. From the realization that a significant body of pre-existing external human knowledge could be represented and usefully applied to user-generated information came the extraordinary new idea of machine proofreading. File transmission applied signal-bearing wires to the space between people, rather than just between devices. Combine data application and wires and you get the net as gigantic reference library and perpetual market. Combine wires and internal state and you get distributed applications and the net as communication infrastructure. Combine data application and internal state and you get data mining and machine translation. Combine all three and you get more or less everything in modern computing up to the night before IM, online dating, eBay, Mapquest, Napster, Google, SETI@Home, "people who bought this also bought", phonecams and the blogosphere.  

But it's the next morning, now, and I don't really want an information appliance. I want a virtual personal assistant. I want my writing software to think of its job not just as formatting documents, but as remembering everything I do when I write, including things I don't realize I'm doing, and things I do while writing that aren't themselves writing. I don't just want document-level Undo, I want a coherent journal view of everything I typed, including all the dead-end phrases I tried and deleted and might now want to revisit. Actually, I don't want fundamentally document-level anything, I want a dynamic evolving history of my entire interaction with my computer and the network beyond it, navigable by chronology or association. I want to jump from an email to the web page on which I found the stat I cited four replies ago in the note that started the conversation. I want to go from the song I'm playing to the birthday of the person who told me about it, to a cross-referenced list of the other music I associate with the song and the other music all my friends have mentioned in emails and IMs and forum notes and shared playlists and now-playing monitors. I want to see rhythms of correspondences and patterns of discovery and contours of neglect. I want the things I've forgotten to know when to remind me of themselves, and the things I think I know to have the humility to volunteer for their retirements.  

The primary challenges for the design of virtual personal assistants are of a different nature (naturally) than the challenges for the design of information appliances or state machines. What the state machine worries about representing, the assistant thinks about communicating and transforming and connecting. What the information appliance struggles to remember, the assistant has to decide how to share and correlate, and when if ever to forget. The state machine works to its capacity. The information appliance works to its parameters. The assistant, however, must be self-governing and evolvingly aware of its own limits, able to differentiate between automating and advising. The assistant will be evaluated not only on what it accomplishes, but on what it knows to ask and when. The state machine's applications were solipsists, however creative. The information appliance's applications were autocrats, however occasionally beneficent or enlightened. The assistant's applications are inventors and ambassadors and advocates and court jesters, and sometimes mercenaries and cannon fodder, and every once in a while oblivious innocent bystanders willing to go home without complaint when you promise them there's nothing to see here.  

And in a connected and definingly social world, the virtual personal assistant is a distributed and intimately negotiated function, and the rules that maintain the productive tension between isolation and aggregation are even more complex. What is the currency of the economy of privacy and trust? On what grounds do you delegate a privilege or retain it? What of yourself are you willing to reveal in return for what collective wisdom, from what collectives, and for that matter which and how much "wisdom" are you prepared to consume, and in what forms? When is it information we seek, and when is information-exchange merely a proxy for personal contact? When does a system become more humane by modeling its users more precisely, and when does it serve them better by leaving them to their own improvisation and compromise?  

The new world will be many things, some of which are already emerging and some of which are yet deeply hidden, but here are a few of what may be its truths:  

- Millions now stored will never be erased. In the last era, everything not saved was instantly lost. In the new era, everything not meticulously preconstructed for disintegration will be indexed and archived forever.  

- Data belongs to people, not processes. There are no silos in the new architecture. Persistence doesn't mean writing something so that it can be reconstructed by its originating code, it means writing it so that it can be reconstructed without its originating code.  

- You are in a maze of twisty passages, each explicably unique and enticingly beckoning. The new systems must not only know when to ask you questions, they must know how to categorize the properties of the possible answers. They must know how to empower your responses with nuance rather than luring you into literalist traps.  

- Everything good is relative. The old era was about identification and instantiation and encapsulation. The new era is about connection and abstraction and subcomposition and change. The old tools had files and records and pages. The new ones have links and self-description and self-direction. The old world was measured in assignments and addresses, the new one in associations and relationships. The old tools took knowledge apart, the new ones must put it back together again.  

- There are three classes of the acted-upon and the acting: objects, creatures and artists. Objects have no value except as they benefit creatures or express the work of artists, and perform no act except in response. Creatures are to be respected and defended and delighted, and acknowledged in their free will, but not burdened with responsibility or solicited for decisions. Artists are the source of all authority and the ultimate ends of all means. Humans are sometimes artists but always at least creatures. Machines and systems and programs (and policies and corporations and governments and precepts, including these) are never more than objects. The first obligation of any designed system is to be obsessively devoted to the intricate cognizance of these boundaries.  

The simplest worthy tools exist to protect or sustain something alive. The best ones express something that makes living more beautiful. What numbers do your machines safeguard that an abacus wasn't sufficient to protect? What do your machines make beautiful, that was ugly when all we had were wood and beads and hands?