In order to improve and enrich our communication and reasoning processes, we should incorporate better structure-expressing conventions and tools, borrowing from quantitative languages and interactive affordances. We should ease up from the strict adherence to inflexible writing conventions that privilege prose over other expressional forms. This expansion of structured expressivity will greatly enhance our general capacity for concrete reasoning.
Definition: [mathematical] structure - the patterns of and relationships between objects of reasoning.
Definition: expression - Something that represents or communicates a thought or concept.
Structured expression therefore means the expression of structured concepts, though the intent is also to connote that the particular visual form of the expression is meant to accurately and faithfully reflect the patterns and relationships within that structure with as much elegance and as little extraneous artifact as possible.
Note to reader: This article is written in an unconventional way using the practices described herein. Click on any of the UI elements below, each of which represent separate sections, to expand them and show their content. It's not necessary to read this article start-to-finish as would be conventional -- use the UI elements to explore the topics that are interesting to you.
This article is structured using some of the concepts discussed herein, in particular the Level Of Detail structure, in which the content could be likened to a pyramid, where the top point of the pyramid is the summary of the whole article, the next layer down provides summaries of the different sections, the next layer down provides more detail for each section, etc. You start by only seeing the top of the pyramid, and you get to drill down into the details only if you see fit -- you "zoom in and out" to see exactly the desired level of detail.
More precisely, the content has a tree structure in which each node's "children" relate to and provide more detail and context for it. The UI presents the tree structure using familiar affordances, with clickable, collapsing/expanding interface elements. This can be thought of as an interactive table of contents that innervates the entire document, going all the way down to the smallest level of detail.
The main body of the article appears in the left column. This right column will present asides and parentheticals so as not to interrupt the flow of the main body of the article. By construction, all material appearing in this sidebar could be considered supplementary and optional.
The specific forms of structured expression used throughout this article should not be taken as a paragon of its practice. The point of this article is to motivate the development and refinement of such practice. I ask the reader to take the steel man approach to this concept and help imagine and converse about what it ideally could become, instead of tearing down its imperfect beginnings.
In fact, this thesis exists because there don't yet exist satisfactory, widespread tools and conventions for structured expression. The intent is for more examples of each kind of structured expression to be added to this article as time goes on. For now, the reader must supplement reality with their imagination.
I've found creating web content using HTML and related tools to be deeply unsatisfactory in the past, in particular the conflation of content and presentation, and I'm finding that to be even more the case here where I'm attempting to provide UI that corresponds to the structure of the exposition. What is presented here in terms of UI should be considered a crude first attempt.
Concrete reasoning process
External: | Initial concrete content | $\longrightarrow$ Ill-defined linguistic process |
Resulting concrete content |
Internal: | $\vert$ Read $\downarrow$ |
$\uparrow$ Write $\vert$ |
|
Internal: | Initial essential concepts | $\longrightarrow$ Cognitive process |
Resulting essential concepts |
This is a commutative diagram illustrating the internal processes vs the externally manifest reasoning processes that define concrete reasoning.*
In the diagram, concrete content could be, for example:
- An essay
- A computer program
- Content on a web page
- A scientific journal article
- A legal document
- A musical score
- A novel
- A history textbook
- Really any kind of document that requires a reasoning process to produce or improve.
Essential concepts are the internal cognitive model of the content. This diagram is said to commute in that the two paths from "initial concrete content" to "resulting concrete content" are posited to produce the same result. In particular, the "ill-defined linguistic process" is defined as the composition of the other three arrows' associated processes:
- Read, then
- Cognitively process (potentially nonverbally), then
- Write.
*From the perspective of an outsider, only the objects and processes in the row marked "External:" is visible, and these are what define the concrete reasoning process. The rows marked "Internal:" are what the person doing the reasoning is doing, potentially with no outward facing indication. These represent the intuitive or abstract reasoning process.
The problem here is that the reading and writing processes in the diagram are highly contextual, subject to interpretation, and are often subjective. This is part of why the ill-defined linguistic process is so ill-defined. By adding forms of structured expression, we bring the concrete content closer to being a faithful representation of the internal, essential concepts. This effectively decreases the "length" of the read and write arrows by not demanding that the essential concepts be crammed into contrived writing forms and conventions that don't suit the concepts. This has several positive effects:
- Quantitative improvement: The concrete reasoning process becomes more efficient and clear (due to the increased similarity between essential concepts and concrete content), especially when involving interactive forms of expression.
- Qualitative improvement: Additional avenues for concrete reasoning open up due to the addition of new forms of structured expression. Without those forms, concrete reasoning about those structured concepts is either difficult and awkward, it's intolerably slow, or practically impossible.**
- Communication of the essential concepts to outsiders becomes more efficient and clear (again due to the increased similarity between essential and concrete).
The combination of the quantitative and qualitative improvements can't be overstated here. Structured expression allows one to reason and communicate about more things, more accurately, and faster. To rephrase what was stated above, the "round trip" that is the concrete reasoning process becomes much shorter, traversing a path that's closer to the essential reasoning process happening internally.
**Consider the transition from Roman numerals, in which addition was difficult and multiplication was practically impossible, to Arabic numerals, in which the addition, subtraction, multiplication, and division of arbitrary numbers can be taught to a generic 10 year old.
I like to say that a natural language is a very bad approximation of telepathy, where telepathy is defined as the ideal of effortless, perfect conveyance of thoughts, feelings, and concepts*** between individuals. While natural languages are poor approximations, it has been effective enough in producing global, human civilization -- a qualitative leap beyond any other species on earth. Writing systems allowed that crude, approximate telepathy to transcend geography and generations; this was literacy. Mathematics introduced quantitative reasoning into domain-specific niches of human discourse; this was numeracy.
The Structured Expression Project is an effort to bring elements of quantitative languages into the mainstream conventions of communication and reasoning. The application to technological contexts is obvious, but the project is intended to apply to humanities contexts as well. The goal is to take one step forward from "very bad" in our approximation of telepathy, and ideally take a qualitative leap forward in our practical, collective cognition.
***What is meant by "thoughts, feelings, and concepts" is really "any possible content of a mind".
Natural languages, along with the attendant writing systems, necessarily render concepts in linear sequences of words, as that is the constraint imposed by human speech. For many common grammatical forms the constraint is perfectly natural, for example subject-verb-object, in which the subject and object are related by the verb, which therefore can reasonably be considered a linear concept. However, the constraint is not conducive to the expression of concepts that have nonlinear structure. To be explicit, to put a nonlinearly structured concept into words is to break its nonlinear structure and choose a particular encoding of its constituent parts into words.
Natural languages are arguably the primary reason that the human race has been so successful. Natural languages provide the mechanism for the development and persistence of culture, civilization, and technology. No individual human is capable of creating modern life truly starting from scratch. To increase our expressive power is to enhance the conduit connecting all of our minds, which is the medium of civilization.
Examples of nonlinearly structured concepts rendered into words:
Dependent clause: "The first power plant built in that country, whose economy at the time was still mostly agrarian, accelerated economic growth in an unprecedented way." This is perhaps the simplest example of a nonlinear concept in language, in which the dependent clause acts as a grammatical "cul de sac" off of the "through road" of the main clause.
Analogy: "A is to X as B is to Y." There are two kinds of relationships here:
- The relationship of A to X, analogous to the relationship of B to Y.
- The pair A and B and the pair X and Y, where each pair corresponds to one context.
Multiple, independent adjectives: "Steve-o bought an expensive, pretentious, garish, red suit." Here, each of the adjectives is independent of the others; they all modify the noun "suit", and therefore the order of adjectives is arbitrary and irrelevant, but are typically governed by language-specific conventions.
- Family tree: "Well, there's me, my mom Kendy, my dad Brudian, my maternal grandma Antopheris, my maternal grandpa Sperrand, my paternal grandma Warshiet, and my paternal grandpa Nart."
- Nesting structures: "On Monday, I'm hanging out with Gionni to take a walk and talk about books, then I've got a dentist appointment; on Tuesday, I'm working all day, then gonna go to Burzber Hall to see The Scinoms and also Hoctone play; on Wednesday, I've got nothing."
The names here were produced by my "made-up word generator". In particular, they were randomly generated from an N-gram based Markov model trained on a list of names, which I'll post about later, and ideally add as an interactive webpage on which you can generate hilarious, made-up words directly, on whatever corpus of input text you like.
Tone of voice -- using a parallel channel (tone) can communicate context.
- "You did such a great job", said earnestly, can be understood to have its obvious, positive meaning, whereas if said with sarcasm, the phrase carries nearly the opposite meaning. In this case, the significant meaning is carried entirely by flouting a principle of cooperative communication.
- A rising voice tone at the end of a sentence often indicates a question. Up-talk is another usage of rising voice tone, which is commonly considered to be annoying or dumb-sounding, but could be considered a form of paralanguage which communicates "I will continue speaking".
Adherence to the writing conventions that prefer paragraphs of prose is an obstacle to accepting additional forms of expression which are more effective for structured concepts that don't faithfully or elegantly render into prose.
Examples, with descriptions of the essential aspects of each as well as discussion of how a breaking from conventional writing forms can improve expression:
The essential aspect of a parenthetical/aside is suggested by the word "aside" itself. The clear way to visually represent this concept is by placing it to the side*, in a separate but parallel body of text, perhaps with an asterisk-type symbol to disambiguate in the case of multiple asides**.
*For example, in a side bar like this one, or in a footnote. However, with interactive visual interfaces, a mouse-over pop-up could be used.
**This is the second aside.
A list of things typically appears as a comma-separated sequence of words with a conjunction (e.g. "and", "or") before the final item in the list. The conjunction indicates the function of the list (e.g. a sequence of possibilities, a sequence of necessities, etc).
Certain essential elements of the list concept are obscured rendering it as a monolith of text:
- The distinct items in the list are not visually distinct; they are embedded within the paragraph and are only separated by punctuation.
- The order of the items is somewhat obscured by the fact that they are "wrapped" into the lines of the paragraph.
- The "lateness" of the conjuction within the list of items.
Obviously the essential structure of a list is
- the distinct items and
- the order of the items.
A simple visual affordance is to render the list as a sequence of bulletpointed items. This way, they flow uniformly down the page, each distinct item having its own, clear line.
The Oxford comma is the sensible convention that retains the comma before the conjunction, so that the final two items aren't parsed as one.
An open question in my mind is what, if any, grammatical constructs should be used along with a bulletpointed list. In this article, I'm playing with different conventions. One is to stay near a grammatically correct sentence structure except that the list items are broken up into their respective bulletpoint items. Another way is to have each item be its own, grammatically self-contained clause.
Furthermore, what convention regarding punctuation and capitalization should be used? The bulletpointed list affordance does the work already, so capitalization and punctuation are not strictly necessary. Though for me, because I was brought up using traditional writing conventions, leaving off capitalization and punctuation feels wrong. I choose to view this as the usual discomfort of learning/exploring.
An argument (logical or rhetorical) consists of a number of statements, each being logically implied by a number of supporting statements, eventually terminating at the foundation of supporting statements, which are taken as assumptions of the argument. Typically one statement is singled out as the main conclusion, and that is meant as the ultimate product of the argument; the body of supporting statements are just the means to the end.
In conventional writing, the statements of the argument would be rendered in paragraphs of monolithic text, each statement being sequenced no earlier than any of its supporting statements. Furthermore, each statement must grammatically indicate which of the previous statements are supporting statements, in addition to the essential logic of how the statement is supported. In this monolithic textual form, certain essential features of the argument are obscured:
- Each individual statement is not an obviously distinct visual element, as each manifests as some portion of the paragraph of text, indicated only via punctuation.
- Given a particular statement, the set of supporting statements is not visually indicated, and must be inferred by parsing the statement and potentially even doing more cognitive work to determine which statements are being invoked as support.
However, the actual structure of an argument is that of a directed, acyclic graph (DAG), in which
- the vertices of the graph are statements, and
- there is a directed edge from statement X to statement Y if and only if X is a supporting statement of Y.
Rendering the argument as a DAG makes it visually obvious what the logical progression of statements is, you just follow the arrows. Visual clarity could be further enhanced if the DAG were arranged such that its arrows were only allowed to point downward (at any angle with a downward component; not horizontal or upward). From this visual representation, the foundational supporting statements would appear at the top of the visualization, and the ultimate conclusion(s) would appear at the bottom. Finally, a use of circular logic becomes visually obvious -- it appears as a cycle in the graph, which would require adding an arrow that goes upward -- a big no-no indeed!
In order to utilitize a concept in a sentence in a grammatically correct way, some "padding" has to be used to formally place it within the sentence, when a shorter indicator would not only suffice, but would improve readability, as it would de-emphasize itself and draw attention to the concept being expressed.
Examples:
Prose | Structured expression |
It should be noted that X. | Observe: X |
To give a few examples, A, B, and C. | Examples:
|
Because of the fact that P, it follows that Q. | Observe: P.
Conclude: Q. |
It is the author's opinion that Y. However, it is a fact that Z. | Author's opinion: Y. Contrasting fact: Z. |
If U, then noting that V, it follows that W. It is an then obvious consequence in particular that X. | Observe: V.
Conclude: U $\implies$ W. |
One more example will be given outside of the table, as it requires more formatting than the table allows. Example of dialog:
Speaker | Emotion | Dialog |
Mr. Knutlesan | incredulous | Really? This is the first time you've ever mentioned that! |
Mr. Vascotten | indignant | Yes, really! |
Mr. Vascotten | subtle tone of threat | You do agree, yes? You wouldn't want to be viewed as negligent, would you? |
Mathematics and computer programming languages are examples of quantitative languages which are designed for the very purpose of representing structured concepts and carrying out the attendant reasoning processes. They each employ their own expressional tools to achieve faithful and elegant expression of the relevant concepts.
Definition: a quantitative language
- has an associated process, which takes
- some kind of input and
- produces some kind of output, that is
- subject to some notion of consistency.
Mathematics | Computer programming languages | |
Input | axioms | human-readable source code |
Output | theorems (true logical statements) | well-formed, machine-executable program |
Process | proving: derivation of formal proofs of theorems by applying rules of logic | compilation: translation of human-readable source code into machine-executable program |
Consistency | theorems must be logically consistent (i.e. mutually non-contradictory) | output of machine-executable program must match the theoretical output of the program as defined by human-readable source code |
Mathematics is arguably the most powerful tool pertaining to the real world that humans have ever devised. In particular, it allows us to model physical reality and even predict, qualitatively and quantitatively, unknown physical phenomena. On the surface, this seems like a rather miraculous fact, since mathematics is effectively a linguistic process in which rules of logic are applied to logical statements, yet it has such predictive power over the real world. Thinking a bit more deeply though, the miracle is an illusion. Mathematics is simply use of an isomorphism between the patterns and processes seen in physical reality and linguistic patterns and processes that obey the same rules. It's through this isomorphism that the abstract process of mathematics has such predictive power within our physical reality. In general, mathematics allows us to reason about anything exhibiting structure.
Mathematics as a "typographical process" appears in Douglas Hofstadter's book, Gödel, Escher, Bach.
My favorite example of a prediction made by use of abstract mathematics is Einstein's prediction of gravitational lensing as a consequence the theory of general relativity. The prediction was confirmed in an astronomical observation during a total solar eclipse on May 29, 1919, and you can confirm it too!
Computation* could be considered to be the concrete manifestation of mathematics in the real world. While mathematics provides powerful theoretical models for physics, chemistry, biology, cryptography, logistics, etc, it is the computational tools which actually construct particular solutions** or instances of those models. For example:
*Computer science, physical computers, and as a practical matter, computer programming languages.
**Unless the model is intrinsically discrete and finite, such as in cryptography, the mathematics uses continuous quantities which can only be approximated by finite-precision quantities in the corresponding computations, simply due to the finiteness of computer resources.
Mathematical theory | Corresponding computation | Practical consequence of computation |
Newton's law of gravitation | Numerical approximation of solutions to gravity ODEs | Ability to accurately predict the paths of planets, moons, and spaceships for the purpose of space flight |
RSA cryptosystem | Difficulty of factoring integers | Infeasibility of determining the private key from a public RSA key, thereby providing a central piece of cryptographic security for internet communications |
Protein folding | Searching for energetically favorable foldings of proteins | Development of new therapies to treat diseases |
Graph theory regarding shortest paths | Dijkstra's algorithm for finding shortest path | Finding quickest route from one location to another in a computerized map |
Computation forms one essential bridge between the abstract world of mathematics and the concrete, real world. It gives us a practical way to use mathematical models which don't provide easy, closed-form solutions, which turns out is most of them. It would have been literally impossible for us to have achieved our current level of technology without computation.
Natural languages could be considered to operate mostly on a subjective and/or metaphorical level, as even the simplest statements are always subject to context and interpretation. Examples:
- "I'm tired."
- "We all must sacrifice sometimes."
- "I'll see you next Tuesday."
Furthermore, words in natural languages have imprecise and/or multiple definitions. This is totally acceptable to humans because we are contextualizing machines, and can cope with its vagueness in most everyday life situations. The vagueness is probably a good (and maybe even necessary) feature for human use in that it makes a language flexible and doesn't impose an intractable logical consistency constraint. The vagueness is almost certainly the foundation for verbal art forms. But for formal reasoning and quantitative applications, the vagueness of natural languages is a distinct obstacle. Attempting to carry out formal reasoning using natural languages, especially in dialog with others who may have different context or have different definitional understandings of words, easily leads to unwitting logical non-sequiturs due to the conflation of concepts by limited or imprecise vocabulary.
Making a precise, literal statement in a natural language is difficult. For example, "I'll always be there for you" has to become some caveat-ridden monstrosity like "If you need help urgently and I'm still alive and I'm not dealing with a more urgent matter of my own definition and I'm physically and mentally able to help you, then I'll help you." Even then, many of the words are still subject to interpretation and context.
The non-requirement of logical consistency is both an advantage and a disadvantage of natural languages, but logical consistency could never really have been a primary "design" criteria for a natural language. Natural languages developed to be suited for human survival and sociality.
Because of the subjective and metaphorical nature of natural languages, it's not surprising that the reasoning carried out early on using natural languages alone (i.e. in absence of formal or quantitative reasoning) produced anthropocentric, metaphorical views of the nature of reality*, views which are certainly biased by our human cognitive processes. Arguably, this should be entirely expected of natural languages, as their development was a survival trait, and survival meant, for the majority of human existence, the functioning of a tribe and its attendant social structures. Accurate, unbiased understanding of the physical world was not of direct use to survival**, as that was not the trait being selected for.
*Legends, mythologies, religions featuring anthropomorphic deities.
The notion of metaphorical truth, which as far as I can tell was put forth by Bret Weinstein, captures an important distinction between the categories of languages. This is in contrast with literal truth, which pertains to the objective world, and which is the domain of quantitative languages.
**Obviously it IS useful to survival for an individual to have an accurate cognitive model for the physical world in so far as that individual's physical interaction with it affects their survival.
Fascinating question: What would the languages and epistemologies of intelligent, non-social animals be like? This is assuming they would even develop language in the first place, which seems less than certain because of language's clear function in facilitating group cooperation. Would the fact that their survival didn't depend on social structures mean that their evolution would be correspondingly less constrained and that they could afford to evolve more accurate conceptions of objective reality?
The adoption of certain forms of expression from quantitative languages will help complement the expressive power of natural languages. This entails changing writing conventions to allow for the use of certain visual affordances and syntactical constructions which represent the structured concepts that don't render well into prose. The particular benefits are, in order from nice to grandiose,
- Visually interesting affordances, each having their own shape and features, will draw the eye in*.
- The essential aspects of the concepts are made visually apparent, making the exposition clearer.
- Spatial representations of the concepts take advantage of humans' advanced visual processing to do more work to understand the content, while simultaneously offloading work from linguistic centers that would otherwise parse the grammatical constructions.
- By using some tools from quantitative languages, we make some progress toward the "literal truth" side of the reasoning spectrum and away from the "metaphorical truth" side typified by purely natural languages.
Note that some (most?) of these new forms of expression won't necessarily have a faithful representation in speech. Though that's to be expected, as the faithful representation of nonlinear conceptual structures requires breaking linear sequencing constraint imposed originally by the fact that our writing systems render speech.
*Contrast this with a monolithic paragraph of text which is visually indistinct and is tiring to the eye.
A richer expressive toolset that accomodates the modern, technologically complex world will necessarily require more interactive affordances, including ones in which choices about how to view the content. Most content, especially data in computer contexts, has many overlapping but distinct structures, each of which should be thought of as a different "lens" through which to look at that content.
This naturally segues to the Structured Expression Project, which includes a personal software project of mine which aims to provide computer libraries and GUI tools for creating, editing, viewing, serializing, etc structured data, but should be interpreted as a broader effort to develop tools for structured expression.
In particular, the goal of my personal project is to free humans from the confines of static, textual representations of structured data on computers. It's surprising how tied to static, textual representations people are, as well as the attendant weaknesses in reasoning about the abstract, essential properties of that structured data. For example, the text-based source code of a computer program versus its abstract representation.
My experience is that most people I've talked to about this consider a computer program to be identical to its text-based source code, presumably because for decades that's the only way it's been represented, even though the particular syntax used in the text rendering of the program is a contrived and non-essential property.
As one's fluency with a broader set of expressive tools increases, one's capacity for concrete reasoning about the corresponding concepts increases. In my experience, my thinking has been enhanced by fluency with the concepts of structured expression and the as-of-yet-limited tools I use to instantiate them. This suggests that structured expression is a complementary cognitive artifact* and the attendant cognitive process I will call structured thinking.**
The process of thinking deeply on a topic often involves trying to untangle concepts and figure out what the essential relationships are so that non-essential artifacts can be discarded. Determining that certain concepts are structured in particular ways can greatly inform one's thinking on the topic, suggesting that the essential features of the structure discovered in the concept can then be applied to the concept. Having a rich tool set for this purpose makes this process all the richer.
*This notion was discussed by David Krakauer in a Sam Harris podcast (search for "complementary cognitive artifact"; it appears about halfway down the transcript) and could be summarized as being a tool which enhances one's practical ability in some way while simultaneously enhancing (not detracting from) one's cognitive ability to perform that function. The opposite concept is that of competitive cognitive artifact, which enhances one's practical ability in some way but atrophies one's cognitive ability to perform that function.
**I believe I first heard this term from Raffi Bedikian while we were talking about the general topic of this article.
Presented here is a first pass at defining a collection of categories that separate out the most salient, independent features of forms of structured expression. This categorization process* isn't easy and may not admit a canonical answer, so I'll mention that it's something to be potentially improved upon in the future.
*I think of this process as analogous to that of matrix diagonalization, in which the goal is to find basis vectors in which the matrix is expressed as simply as possible. This is analogous to as picking concept features which are uncorrelated, whereby the features provide mutually independent information.
The kind of categorization being discussed here is a linguistic-semantic categorization, and so involves imprecise and subjective linguistic reasoning, but I think it would be super interesting if it were possible to do this sort of thing in a quantifiable way, say where some metric like "linguistic simplicity of description" were used to define the quality of a particular categorization. Perhaps there is some structure and theory in linguistics unknown to me that would turn these sorts of processes into a computational process.
The manifestation of a particular form of structured expression informs which media the form can be put into. It's possible that a form of structured expression has both static and dynamic manifestations, and potentially multiple of either.
- Static: Manifestation of the form is static and unchanging. This suitable for any media, but is particularly suitable for physical print.
- Dynamic: Manifestation of the form may change* based on the parameters controlling its presentation.
*This isn't to say that a dynamic form can't be rendered into a static context. For example, the form could be printed in one particular configuration using a fixed choice of the parameters controlling its appearance.
The interactivity of a particular form of structured expression determines if the form has an interactive interface. It's possible that a particular structured expression can take both interactive and noninteractive forms, and potentially multiple of either.
- Interactive: Representation or use of the form has an interactive interface. Some forms may not have a practical noninteractive form, so their interactivity is required. Examples:
- A very high resolution image, such as a detailed map of the globe, is far too large to be displayed without some interactive element which allows the user to control the view area and zoom level, as well as what information to display, such as roads, topography, territorial borders, satellite imagery, etc.
- Some structures are intrinsically high-dimensional, and therefore admit no practical, static representation as a whole, so they require interactive affordances for the user to determine how to slice and view the content. Examples*.
- The Level Of Detail structure is best expressed interactively so that viewing and navigation is efficient, instead of requiring the user to skip ahead to a particular page as in a conventional Table Of Contents.
- Noninteractive: The form has a representation without any way to actively modify the presentation. This is not to say that the form isn't dynamic, for example, a very high resolution image in which one could zoom and scroll, without actually changing the presentation.
*Examples:
- A massively connected graph, such as a social network
- Highly parallel time series data, such as a conductor's score of a many-voiced orchestra
- Volumetric data, such as in fMRI imaging
- Samples in a high-dimensional feature vector space, such as public officials' voting records or feature hashing (e.g. as done by Shazam)
The parametrization of a particular form of structured expression is whether or not there are any parameters which govern the form's representation, or if there is essentially only one way to display the form.
- Parametric: The form has a number of parameters which determine how it is rendered*. Each unique combination of valid parameters gives a distinct representation of that form -- though the content being displayed is not changed. Most forms of structured expression admit at least some parameters.
- Nonparametric: The form essentially only has one representation that doesn't alter the essential structure. Examples**.
*This refers to the view component of the model-view-controller design pattern.
**Examples of non-parametric forms:
- An interactive map only showing satellite imagery, or this image of central parts of the Milky Way galaxy, which is too detailed to view all at once, and so the user must scroll and zoom to choose which parts of the image to explore and view in detail. Link source.
- A list, if the given order is crucial information
I'm not 100% sure that this is a useful category, since most forms of structured expression admit some possible parameters. Perhaps what would be more useful is just identifying what the various parameters are for a particular form of structured expression. Even in the case of an ordered list, the list could be displayed vertically or horizontally, or collapsed into an abbreviated form given an interactive interface.
The availability of tools informs which particular forms of structured expression are practical to use at the current time. As more tools are developed*, the range of feasibly usable structured expression grows. Ideally, this category will eventually disappear as tools are developed to cover everything.
- Current: There are well-functioning writing conventions or software tools, currently available to use this particular structured expression.
- Future: The writing conventions or software tools for this particular structured expression are not well-developed or are non-existent, and therefore it's impractical to use this particular structured expression.
*Tools, here, refer to both writing conventions and software to support a particular form of structured expression.
The concrete vs abstract distinction, also called the type-token distinction, as it pertains to expression is really noticing that a concept typically has concrete representations in writing, speech, or in visual or auditory forms, but those representations aren't the same as the concept itself. The concept has an abstract, essential form that's independent of any possible representation. The abstraction can be thought of as the relationships and patterns defining or governing that concept. A representation of that concept is a human-created form. Always remember that a concept's essence is distinct from its representation.*
To rephrase more strongly: It's a cognitive trap to conflate the concrete representation of a thing with its abstract essence!
This relates to the model-view-controller architectural pattern in user interfaces in that it is a separation of the important facets of the concept or data:
Model | Abstraction | Strictly limited to defining essential features of the concept/data |
View | Concrete representation | Strictly limited to how the concept/data is presented (typically visual, but could be auditory, etc) |
Controller | Interactivity | Strictly limited to how the model and view are changed (this component is omitted for non-interactive forms) |
Examples of the model-view-controller architecture pattern (which itself is just a manifestation of the abstraction vs representation concept):
- treevis.net -- shows myriad gorgeous ways a tree can be viewed; model is simply a tree structure.
- Color picker -- shows many views of the selected color at the same time; model is simply an RGB color.
- Tree view -- shows nodes of tree structure whose nodes present as expanding/collapsing UI elements. The model is a tree structure.
- Piano roll -- shows a time series of notes on a grid corresponding to keys on a piano. The model is a musical composition, e.g. a MIDI file. Contrast this with showing a more traditional western musical score.
- File manager -- shows a visual representation of the filesystem; files, directories, devices, etc. The model is a formatted filesystem. Here is another visual representation that uses 3D spatial affordances to represent elements of the filesystem.
- Graph -- shows a discrete graph in many forms (direct visual representation, adjacency matrix, adjacency list, edge list), complete with interactive editing capabilities.
- ADTeach -- an educational tool for teaching about abstract data types and algorithms by linking program source code to visual representations of what's happening. Their paper explicitly mentions the use of model-view-controller architecture in their software.
Use of the model-view-controller pattern should be considered practically mandatory for design of most forms of structured expression, even if the form isn't interactive -- it's an invaluable practice to separate abstraction from representation in order to clarify just what the structured concept really is on an essential level.
One obvious advantage to the model-view pattern** is that it allows a multitude of views to be made without any need to change the underlying model, and that those views can be switched between instantly. This is a huge qualitative leap in the user's power to create, view, and modify structured expression over forms of expression which conflate abstraction and representation***.
A point that can't be overstated is that in order to realize the potential power and efficiency of structured expression, something like the model-view-controller architecture must be used. This allows a person creating structured content with it to focus on the details of the content itself, instead of worrying about how the content is formatted, or worse yet being forced to encode the content in some other format. The view and layout of the structured expression should ideally be independent of its content.
For example, writing this document was a frustrating experience, because it involved encoding the various structured concepts into HTML and shortcodes, making it necessary to worry about presentation at the same time as content, and worse making the editing process of restructuring the presentation to better reflect the essential structure of the content difficult and ponderously slow. Good tools for structured expression would mean, in particular for this document:
- The LOD structure would have distinct tree nodes that could be created, modified, deleted, or moved around to different parts of the tree, and doing so would not involve manually typing HTML or shortcode, but rather using an interactive UI to appear to edit the nodes directly.
- The presentation of the LOD itself could be controlled by the user, presenting in depth-first-order as in traditional exposition, in breadth-first-order to give better and better approximations of the whole content with each section, or using the map style, in which nodes are placed as continents, countries, cities, neighborhoods on a map, and more detail can be obtained by literally zooming in to see them.
- Various repetitive jargon (e.g. "structured expression") could be automatically abbreviated and disambiguated using a mouse-over tooltip UI element.
- The side bar would be automatically generated from the footnotes and asides, each of which would be their own distinct data element. The side bar could be shown or hidden using a UI element.
- Many examples of structured expression could be created quickly, since it would just involve creating their content, not worrying about how they'll be displayed -- how to display each is either chosen automatically or by the user.
- An index and glossary could be automatically generated from annotated phrases (I didn't bother with this because it would be too ponderous with the tools I'm currently using). Glossary item definitions could be shown in-line in the content itself using a mouse-over tooltip UI element.
*Examples of cognitive errors resulting from conflating representation with abstraction:
- Thinking that a computer algorithm is the source code that implements it. The algorithm has an independent existence independent of any particular programming language and especially of any of a programming language's contrived syntax. An algorithm is in essence a well-defined sequence of mathematical operations for producing a particular output from a given input.
- Caring about source code formatting conventions (e.g. indentation, variable naming, bracket cuddling). Instead, it should be observed that these are arbitrary, have nothing to do with the program structure itself, and could easily be handled by the view of a model-view-controller architecture as in projectional editing.
- Belief that linear algebra is generally defined as operations on matrices of numbers. Instead, it should be understood that linear algebra regards abstractly defined quantities, only one category of which is matrices of numbers.
- Adherence to categorization: In accepting that given categories (which in this case are the representation of a particular facet of a concept) are the only possibilities, one loses the ability to reason outside of that constraint. For example, viewing the siloed departments of a university as a legitimate categorization of human endeavor, and therefore being constrained to limit one's career to one such silo.
**Omitting the controller part of model-view-controller as it is unnecessary for this point.
***Examples of forms of expression which conflate abstraction and representation, which in particular do not practically exhibit the model-view-controller pattern:
- Traditional writing conventions (any structure is irrevocably tied to the particular encoding into paragraphical monoliths of text)
- Programming languages using text-based source code (this conflation is so ingrained in our software culture that this is nearly all programming languages)
- HTML-based content, including this document (this has been very frustrating)
- Spreadsheets (while format might be used to indicate the content's structure, it isn't formally recognized or encoded)
Linguistic determinism is not being claimed here. What is being claimed is that expressive tools can enhance concrete cognitive processes -- the reasoning processes that manifest in and about the real world.
It is not a goal of this thesis to propose converting our writing practices fully into mathematical, quantitative, or symbolic exposition, as this would give up the human-oriented tools* that our languages provide us to function in our social contexts, such as narrative or conversational writing. Rather, it does propose allowing for and encouraging conventional prose to be sprinkled with other expressive affordances that communicate concepts more directly and enhance readability. In summary, it proposes to use the right expressive tool for each job.
*Human-oriented tools such as metaphor and context-requiring vagueness.
Note that the way that the examples are presented here is far from ideal. Many are links to static content on Wikipedia, or are presented statically in-article using HTML-based elements. This gives a rather poor, manually-driven, and somewhat colorless exposition. An ideal presentation would involve rich, interactive views of all the examples, where the content could be interactively viewed, navigated, edited, and created using the relevant structure itself (instead of having to write e.g. HTML code to indirectly represent it).
Some resources showing examples of rich visualization of the kind that would ideally be used in structured expression:
- The Data Visualisation Catalogue -- an excellent resource which lists a huge number of visualizations for numerous structured concepts. The visualizations aren't interactive, but it's not hard to imagine how they could be made interactive; clicking on various components to see more detail, or to change to a different view in which other structure is shown
- treevis.net -- tree structures are ubiquitous, and this site shows a huge number of really pretty visualizations for trees.
- visualgo.net -- some examples of interactive visualizations are given here, though the interactivity for some examples is less than optimal.
- Data visualization -- Wikipedia article on the overall subject.
The reader is invited to imagine to their own satisfaction* such interactive tools that would allow direct use of structured expression in content creation as well as content viewing.
Eventually, as the tools for structured expression are developed and refined, this article will use them to advance toward that rich, interactive ideal of presentation.
*The sense of "imagine" here is to envision what all the different, ideal possibilities could be, in a way that's compelling to the imaginer. At the time of this article's initial writing, the incompleteness/unavailability of good tools of structured expression make this mental work on the part of the reader necessary.
This also relates to my opinion that in a good narrative (such as the proposal of this thesis) leaves some details to the imagination of the reader. These are to be filled in with details and conjectured possibilities that are far more compelling to the reader's conception of the fictional world than any complete description by the author. A story that prescribes too much makes that fictional world finite and futureless, limited by any lack/failure of the author's imagination.
I'm not claiming to have invented any of these structures or expressional forms. I'm just synthesizing them into a rich conceptual system and set of tools, as well as trying to make improvements in doing so.
Level Of Detail is used in interactive maps and game engines to render visible, spatial objects with a level of detail appropriate to their distance to the viewer. Think of being able to "zoom in" to take a closer look at the details of the specific things you're interested in. Examples:
- This article itself presents as an interactive LOD, provided by the collapsing/expanding UI elements.
- An interactive map which allows you to smoothly zoom to any level of detail -- globe, continent, country, province, city, neighborhood, street.
- A Table Of Contents (TOC) of a printed book is a primitive form of non-interactive LOD, in which the tree nodes are presented in depth-first order. Wikipedia generally presents a hyperlinked TOC for any page that is divided into sections -- this is a primitive form of interactive LOD.
- Computer graphics, especially games, uses LOD in improving rendering efficiency.
- A file manager presents the contents of a computer filesystem via its directory hierarchy, typically offering a view in which directories can be expanded/collapsed to view/hide their contents.
- Computer program source code editor, in which hierarchically organized code blocks can be expanded/collapsed.
The LOD concept generalizes from spatial content to expositional content in that the reader should be able to select the desired level of detail at which they engage the material, "zooming in" to see specific details. Ideally, this is presented as an interactive affordance.
There are several advantages to the [LOD] tree structure over a conventional, flat/linear structure:
- Hierarchical organization of material -- contrast with a non-hierarchical sequence of sentences or paragraphs.
- Efficient navigation of material -- in the language of complexity theory, one can traverse a balanced LOD tree in $log(n)$ time, where $n$ is the number of nodes in the tree. Contrast with having to perform a linear search through a sequence of sentences or paragraphs.
- Compact presentation -- the only thing visible is the information that is explicitly requested by use of the expand/collapse UI elements.
Because of the interactive affordance that hides detail unless explicitly asked for, there is less of a need to stay constrained to one point or to a linear reasoning sequence, as would be recommended in a traditional writing convention. This is a double-edged sword, in that the author can present as many rabbit holes as desired and the reader can choose which ones to go down, but at the same time it provides more opportunities for the exposition to undergo scope creep.
In non-interactive contexts, such as in physical print, a static manifestation of the tree has to be chosen. A nice set of static representations of trees is given here. Two notable static representations are:
- Depth-first traversal order -- this corresponds to how a conventional book is organized. Its Table Of Contents provides a rough tree structure at the beginning of the book, but then the contents of the book are enumerated in order, starting at the beginning of chapter 1, and proceeding at the maximum level of detail. While printed books are invariably structured using this convention, reading start-to-finish is not an efficient way to gain an understanding of the content.
- Breadth-first traversal order -- this doesn't really appear in conventional writing formats, but is more analogous to an interactive map. The distinct advantage to this form is that the reader can simply read from the beginning and after completing each level they will have a complete picture of the content to some level of detail. Analogy*:
Level Map Article 1 Globe Overall thesis statement 2 Continent Rough supporting statements 3 Country Sections detailing each statement 4 City Subsections with examples, more detail 5 etc. etc.
*The specific elements given in the table under Article aren't a strict prescription. A document can use the LOD to structure articulation of detail however desired, although there is something more elegant about the different sections of details given at each level being on par with one another, analogous to a country appearing at approximately the same zoom level as other countries, a city appearing at approximately the same zoom level as other cities, etc.
An interesting example of a static LOD is Wittgenstein's Tractatus Logico-Philosophicus, in which each statement is numbered and put into an LOD structure. Here is a visual representation of its LOD structure in which the levels are given as columns, with nested items aligned in such a way to indicate the tree structure.
Often times there will be more than one channel of exposition presented in parallel. The word "parallel" implies that there is some correspondence between elements in the parallel channels. Presenting the channels each as a vertical column in which its content is laid out is a natural and faithful way to visually represent the channels, where corresponding elements are indicated by being placed in vertical alignment with one another. In an interactive context, the user should be able to hide/show each channel independently.
Examples of parallel channels of exposition:
Channel 1 | Channel 2 | Channel 3 | Correspondence |
This article's main body | This article's sidebar content | N/A | Sidebar notes correspond to the statements they contextualize |
Body of text | Text's translation into language X | Text's translation into language Y | Simplistically, each single sentence in the original body of text corresponds with exactly one sentence in each of the translations |
Source code of a computer program | Compiled assembly language instructions | Profiling data showing how much cumulative time was spent executing each asm instruction | Sequences of assembly instructions correspond to each single line of source code |
Source code of a computer program | Human-friendly pseudocode | Comments and other documentation | Lines of source code correspond to lines of pseudocode and to comments |
Version of a text file (e.g. source code) | Another version of that file | Maybe yet another version of that file | Side-by-side diff -- corresponding lines are determined and differences annotated |
Timestamp of most recent edit | Author of most recent edit | Source code | Each line of source code is annotated with its last modification time and author, as in git blame |
Timestamped log events in Stream 1 | Timestamped log events in Stream 2 | Timestamped log events in Stream 3 | Timestamps appear in order and equal timestamps correspond to one another |
Human-written narrative for a scientific/technical exposition, providing the history, motivation, context, and reasons for the significance of the result | Analysis of the result or data | Raw data, analysis algorithms, etc. | The correspondence between the channels would depend on the field of discourse |
Human-written narrative for a mathematical exposition, providing the history, motivation, context, and reasons for the significance of the result | Informal proof of result (written by expert, requiring expertise to read) | Formal proof (computer verifiable) | Each element of the informal proof would correspond to many elements of the formal proof, and sections of the proof(s) would have corresponding written narrative |
I think that the mathematical exposition example is one of the more interesting forms of structured expression geared toward technical exposition, as it confers the following qualitative advantages over the conventional academic paper format:
- A formal proof of a mathematical result is provided in what's basically a computer programming language and can be verified by computer with no expert reviewer needed. This eliminates the need for a referee to manually check the proof. It also elevates the rigor of the exposition from "acceptable to an expert in the subject"* to objective, mechanically verifiable, and indisputable.
- Given that the formal proof is valid, the "interesting" part of the exposition with respect to publication in a journal is really the narrative it provides. It contextualizes the result, establishes why it's interesting within the field, describes relevant history, mentions some related open problems, and generally provides the "human" side of the work. The separation of this from the informal and formal proofs makes it so that a non-expert could read this narrative and understand as much as they possibly could have, without wasting time skipping through the parts of a conventionally-formatted paper that present the proofs or other technical details.
- The informal proof (i.e. the human-written proof in conventional journal article) can concern itself with only the technical details without any extraneous fluff getting in the way. In my experience, having to intersperse proofs with narrative is annoying and what is considered good exposition depends unstably on the referee's unknown/arbitrary opinions about writing.
- Different mathematicians might be better at the work in the narrative channel, human-written proof channel, or formal proof channel, and the explicit separation between the channels gives a clean delineation between the different types of work. This gives another kind of specialization which can allow better collaboration between mathematicians, each one focusing on their strength. Examples**
*What suffices as proof in mathematical exposition currently is really a social construct, defined as whatever the relevant practitioners consider to be an acceptable proof. Humans are great at interpreting, filling in omitted details, and contextualizing, but in the context of research-level mathematics (or any field), the context needed is provided by a PhD-level education in the relevant field.
**Examples:
- One mathematician might be purely interested in formalizing results in mathematics by generating formal proofs and ideally adding them to the public corpus of known, formally proved results; Kevin Buzzard is one such mathematician, in particular using the Lean theorem prover system.
- Another mathematician might be a highly gifted technician regarding producing conventional, human-written proofs, and has little interest in the narrative or formal proof channels of work. This mode of work could be considered to be the main activity of professional academic mathematicians.
- Yet another mathematician might be talented in understanding the broad landscapes of mathematics and providing the context and background that ties it all together into a coherent and compelling narrative; Marcel Berger was one such mathematician, in particular having produced A Panoramic View of Riemannian Geometry.
The idea here is that in most content, there will be many different structures present, describing different aspects of that content. Each structure will have its own visual representation, and the representation of a particular structure could be thought of as looking at the "projection" of the corresponding aspect of the content. Examples:
This may be surprising to some, but I view structured expression as just as applicable to the humanities as to technical disciplines.
In particular, given any story involving characters, there are many implicit structures present within the story's content which relate to the complexity of the characters' lives and interactions. Most, if not all, of the structures listed here could be extracted, annotated, and curated by an editor independent of the author. It's possible that AI could eventually automate this admittedly tedious process.
Regardless of how it's produced, this annotation could potentially be useful to the author in understanding the many facets of the content's structure, and perhaps even find what would otherwise be difficult-to-find patterns and make conclusions about the content, especially in the context of creating a historical narrative.
Different structures within a story:
- The story itself, in conventional text form
- The sequence of the parts of the story in which two given characters (countries, companies, entities, etc) interact* -- this could be thought of as the interaction index for the story.
- In an interactive environment, the interaction index could be searched/filtered, so that the reader could search for exactly content they want, and by clicking on a particular element from this sequence, they could navigate precisely to the annotated section of text representing that interaction. The section could be highlighted as a visual affordance.
- The set of relationships between characters, annotating which parts of the story correspond to which relationship, perhaps containing dates for the start of, end of, or change in the relationship -- this could be thought of as the relationship index for the story.
- As in the case of the interaction index, in an interactive environment, the relationship index could be searched/filtered, and clicking on an entry would navigate the reader to the main text, appropriately highlighted.
- A graph representing the network of interactions/relationships between all characters. This might be a very dense network, so efforts to manage the complexity might have to be taken, requiring an interactive environment:
- The reader could "focus" on one particular character, which would cause the view to center on the node representing that person, and turn the graph into a radial graph. This emphasizes the clarity of visualization of the focused person's interactions/relationships.
- The reader could click on a particular edge representing the interactions/relationship between the two characters it connects. This would bring up the filtered interaction/relationship index in which the sequence of interactions between those two characters is shown.
- Some characters may have very many connections**, so their social network would be very dense. If the interaction/relationship index had chronological annotation, then one of the following could be done to ameliorate the visual complexity:
- A limited time range could be selected for viewing, so interactions/relationships that fall outside of that time range are not depicted in the graph.
- The 2D graph could be extended into 3D, where the third axis becomes time. Imagine each node in the graph now becoming a column extending up and down forever, and the edges which represent the social connections stay as 1D lines***, positioned vertically to indicate their chronology. Furthermore, the frequency of interaction between the selected character and the other characters over time could be used within a force directed graph scheme in order to show each character moving in and out of the selected character's orbit over the course of time, thus keeping each segment of time relatively uncluttered. Here is a 2D version of what I've just described, in which time proceeds horizontally, and the connections are displayed vertically.
- A flow chart**** representing the overall structure of the story, with character interactions, turning points, and major events in the story. This is a way to look at the story as a collection of parallel channels, and can give the reader a big-picture, nonlinear view of the story (contrasted with the conventional start-to-finish, linear reading of the story in its every single detail).
- The sequence of utterances made, actions taken, events participated in, etc by each character -- this could be called the character index. Analogously to the interactions/relationship index, this could be searchable/filterable.
- The set of locations within the story -- the other annotations (interactions, utterances made, actions taken, events participated in) could provide the associated location, giving another view into the story -- call this the location index. For example:
- A visual map (literal in the case of real locations or symbolic in the case of some unspecific locations within a fiction) in an interactive setting could provide a way to search/filter all location based events in the story, bringing up the sequence of events associated with that location. As in the other index affordances, clicking on a particular event would bring that section of the text up, highlighted to indicate the exact annotated section.
*Ideally, every single interaction between any two (or more) characters would be annotated, giving the complete character interaction structure for the story. Better yet if the date/time of the interaction is also recorded, giving the index a precise chronology. Furthermore, the location of the interaction could also be annotated, giving a spatial element to the story's structure as well.
**For example, the subject of a biography, or a character whose whole life and whose totality of relationships is captured within the book.
***For a somewhat more advanced, but more faithful representation of the time range of each interaction/relationship, the 1D edge in the original graph could be extruded to be a 2D rectangle connecting the columns of the two characters, where the height of the 2D rectangle indicates the time range of the interaction/relationship.
****So far, the best I could find as an example of a flow chart for a book one which gives a generalization of the structure of novels.
As an aside, I think it would be a great proof of concept to create the annotations and indexes listed here for some existing, well-known stories to see exactly how these views of the content manifest, and what sorts of distillations of the stories it provides.
- The results of natural language processing on the dialog of each character, for example:
- The estimated reading/vocabulary level of the character, perhaps estimating age from this
- Analysis determining the psychological makeup of the character
- Analysis determining the nature of interactions/relationships between characters
- An analysis of correlations between quantifiable structures in the story, such as:
- The clustering of characters based on their interactions/relationship graphs
- A quantification of the importance of each character or event in the story
- A histogram representing the locale of each character over the course of the story
- The role of each location in the story, perhaps as having to do with some classification of the events happening there
While this sort of analysis is not really structured expression, it is a natural direction to go using the qualitatively greater power provided by structured expression in this context (i.e. not having the various interactions/relationship/character indexes basically precludes this sort of analysis entirely).
The source code of a computer program has many structures, each of which would be a frequently useful tool for navigation and editing of the source code. This is closely related to (and in some cases coincident with) projectional editing.
- Abstract syntax tree of the source code
- Hierarchy of namespaced symbols
- Symbols could be further grouped by which translation unit they belong to
- Type/class hierarchy (which would actually be a DAG if types can have multiple supertypes)
- Function call graph -- a directed graph showing which functions call which other functions
- Code dependency graph -- showing which symbols are depended upon by a particular symbol
- Given profiling data, some visual representation of the performance of each function, e.g. as in KCacheGrind (this screenshot itself shows three different visualizations of the profiling data)
Consider a huge music database like iTunes or Spotify. The basic content is zillions of digitized songs. However, there are many other structures present which organize the content. Structures that are present:
- Set of songs in the database
- Hierarchy: genre $\to$ artist $\to$ album $\to$ song
- Sequence of the set of albums released each year
- Songs ordered by popularity (likes); songs by a particular artist, ordered by popularity
- Histogram of play count per song; histogram of play count per song by a particular artist
- Weighted graph representing the similarity between artists or individual songs
- Quantification of a listener's musical tastes, used in the context of song recommendations, i.e. "your musical taste is similar to this other person -- they like this song you haven't heard, so you might like it too"
The process of browsing web content clearly has web pages and their hyperlinks as the basic content. Structures that are present:
- The basic content -- web pages and hyperlinks -- forms a directed graph structure (hyperlinks don't generally have references back to the referrer page)
- A user's browsing history -- a user can "go back" and then take a different "fork" in the road, which produces a tree structure.
- User's bookmarked pages
- As a chronologically ordered sequence (based on time of bookmark)
- Hierarchically organized based on user-defined folders in the saved bookmarks
- Hierarchically organized: web site domain $\to$ bookmarked page
- User-curated collections of pages, organized in whatever structure is desired, for example:
- A sequence of pages which teach some body of knowledge
- A set of news articles all reporting on the same event or related events
- A hierarchy of pages, where the hierarchy structure comes from the categorization of the page content
- A collection of pages which, together, provide a narrative on a particular topic
This particular example came directly from conversations with Raffi Bedikian during our time at Leap Motion, working on a hackathon project which we envisioned would eventually include this sort of multi-structure approach to navigation of the web and one's local filesystem.
A table is a multiply-indexed data structure (here, "index" and "parameter" are synonyms). For each valid combination of the specified table parameters, an outcome is specified. This concept is the direct product of the index sets, which can be thought of as a grid of all combinations of the index values. In certain situations, not every possible combination will be given.
There are two common ways a table is laid out, but generally it always involves a grid-like visualization. One visualization directly represents the direct product structure, in which each index set is assigned an axis (vertical or horizontal), and the values are enumerated along each, and their combination forms a grid. The grid is populated with the content of the table. Examples of direct product style tables:
- Multiplication table -- the two parameters are the inputs to multiplication, which are presented along the horizontal and vertical axes of the table, and all possible combinations of inputs manifest as the inner grid.
- Matrix notation -- the index sets (the row and column numbers) are usually not displayed, since they're just integers starting at 1.
- Categorization, for example:
- Partial categorization of forms of structured expression
Non-interactive Interactive Non-parametric Bulletpointed list Level Of Detail article Parametric Sorted set of items in print with fixed sorting criteria Sorted set of items with GUI to choose one of many sorting criteria - Examples of tensors -- the index values are the number of covariant and contravariant tensor components.
- Partial categorization of forms of structured expression
- Karnaugh map
If a table has more than 2 indices, then its visualization is a little trickier, and there are different ways to achieve visualization. One common method is where each element of the table is displayed using an entire row within a grid, and that row explicitly shows the index values as well as the "outcome" value. Unfortunately, this breaks the visual representation of the direct product structure. Examples of row-based tables:
- Relational database table
- Truth table -- for example, this logical AND truth table
A way to handle tables with more than 2 indices while still attempting to represent the direct product structure visually is to use nested grids of grids. For example, a truth table for a function of 4 boolean values $f(P,Q,R,S)$ could be rendered as:
$R=F$ | $R=T$ | ||||||
$S=F$ | $P=F$ | $P=T$ | $P=F$ | $P=T$ | |||
$Q=F$ | $T$ | $T$ | $Q=F$ | $T$ | $T$ | ||
$Q=T$ | $T$ | $F$ | $Q=T$ | $F$ | $F$ | ||
$S=T$ | $P=F$ | $P=T$ | $P=F$ | $P=T$ | |||
$Q=F$ | $T$ | $F$ | $Q=F$ | $T$ | $F$ | ||
$Q=T$ | $T$ | $F$ | $Q=T$ | $F$ | $T$ |
The equivalent table rendered as rows would be:
$P$ | $Q$ | $R$ | $S$ | $f(P,Q,R,S)$ |
$F$ | $F$ | $F$ | $F$ | $T$ |
$F$ | $F$ | $F$ | $T$ | $T$ |
$F$ | $F$ | $T$ | $F$ | $T$ |
$F$ | $F$ | $T$ | $T$ | $T$ |
$F$ | $T$ | $F$ | $F$ | $T$ |
$F$ | $T$ | $F$ | $T$ | $T$ |
$F$ | $T$ | $T$ | $F$ | $F$ |
$F$ | $T$ | $T$ | $T$ | $F$ |
$T$ | $F$ | $F$ | $F$ | $T$ |
$T$ | $F$ | $F$ | $T$ | $F$ |
$T$ | $F$ | $T$ | $F$ | $T$ |
$T$ | $F$ | $T$ | $T$ | $F$ |
$T$ | $T$ | $F$ | $F$ | $F$ |
$T$ | $T$ | $F$ | $T$ | $F$ |
$T$ | $T$ | $T$ | $F$ | $F$ |
$T$ | $T$ | $T$ | $T$ | $T$ |
A flow chart presents a visual representation of a decision process or other causal structure. Examples:
- Algorithm
- Decision process
- Understanding flow charts
- Tech tree in a strategy game
- Representation of a novel, history, or a game's story design*
- A choose your own adventure book
Abstract data types (ADTs) could be considered to be the generic building blocks of the structure that structured expression is meant to deal with. Each ADT has its own essential properties and with those properties natural ways to represent them visually. Distinct ADTs' natural visualizations may be similar or even the same due to the similarity between the essential properties of those distinct ADTs. As with most structures, there will typically be many ways to visually represent each ADT.
Because each ADT has its own natural visual forms, compositions of ADTs have natural visual forms which are the composition of their component visual forms. In other words, because the composites are inductively defined, so are the visualizations. The following table gives a commutative diagram describing this relationship. When dealing with generic ADTs and their compositions, this relationship already provides you with a rich toolset for visual representation.
ADT | $\rightarrow$ composition |
Concept |
$\vert$ visualization $\downarrow$ |
$\vert$ visualization $\downarrow$ |
|
ADT visualization | $\rightarrow$ composition |
Concept visualization |
However, in specific contexts, where the structure in the relevant concepts goes beyond their mere definition as compositions of ADTs, the visualizations will have corresponding, additional structure, and the visualization of that additional structure will provide the visualizations specific to that context. Examples:
- In computer programming languages, the abstract syntax tree (AST) is the basic structure of programs, but there is much more structure present beyond the AST, in the constraints of the programming language and its semantics. For example, semantics regarding symbol definition/resolution within a space of nested scopes, or the uniqueness of global function names. Symbols in nested scopes have a tree structure, whereas the unique global function names would have a set structure.
- A spreadsheet program's basic structure is a grid of cells, but often times cells will be grouped together and used to define certain computations. The nature of those computations is the additional structure, which may have different forms, such as a flow chart or a DAG.
- A music tracker program's basic structure is patterns of notes to play using recorded sound samples at given pitches. The additional structure is the arrangement of those patterns to form the full song, e.g. the AABA song form.
Presented here are a number of important ADTs. This is not a comprehensive list by any means. More examples are given here, though this is also not exhaustive.
There are many subtypes of graphs, each with their own natural visual representations:
- Undirected graph; examples:
- a simple, generic graph, demonstrating the generic concept plainly
- a non-uniform mesh, for use in numerical, physics simulation
- Visualization of part of the internet, showing how beautiful and rich a visualization can look
- Directed graph; examples:
- a sociogram, indicating seating preference in a 2nd grade class
- a social network, indicating social connections and degree of connectivity
- a flow chart defines the flow of a set of logical instructions
- a finite state machine defines the behavior for a simple type of machine
- Directed acyclic graph (DAG); examples:
- an argument (logical or rhetorical), where statements are nodes and the support of one statement by another is indicated by an arrow toward the supported statement
- A partially ordered set (poset)
- A type/class "hierarchy" in which each type/class may have more than one supertype/superclass (this is a kind of poset)
- A tree structure is a DAG in which each non-root node has exactly one arrow pointing at it
- A family tree is not really a tree (each person has two* parents) but rather is a DAG**
*A family tree could be taken to describe genetics, in which each person has two parents (excepting the recent development of three-parent babies), or it could be taken to describe the people involved in raising each person (in which case each person could have any number of parents, in principle including zero).
**That a family tree DAG is acyclic is assuming that there no time travel is involved 😉
A sequence is perhaps the most basic of all ADTs, simply presenting a collection of elements in a particular order. This is a very natural ADT within human reasoning because of the strict ordering that our speech, and perhaps more importantly our perception of time, are constrained by.
There are many forms of sequences, and a good number of visual representations for them. A sequence may or may not have a well-defined number of elements.
- Sequence - only really specifies that the collection of elements has a well-defined order, given as an index-mapping from the natural numbers to the elements.
- Array - this has more to do with computer representations of sequences and how they're laid out in memory, usually as contiguous elements.
- Multi-dimensional array - informally, these are grids of elements. Tensors are typically represented in computers this way.
- Circular array - additional structure here is of the sequence repeating after a finite number of elements.
- Linked list - in contrast with array, elements don't have to reside contiguously in memory.
- Queue - additional structure is that there is a "front" and a "back" to/from which elements can be added/removed. Think of a queue of people lined up at a ticket booth.
- Stack - similar to queue, except the metaphor is of a vertically stacked collection of elements, and therefore only the top of the stack is accessible.
Visualizations should be faithful to the defining feature of sequences -- that of the order of the elements. There are many ways, some taking advantage of additional structure present in the particular type of sequence:
- In-line; The first 5 prime numbers are $(2, 3, 5, 7, 11)$.
- As a bulletpointed list; The first 5 prime numbers are:
- $2$
- $3$
- $5$
- $7$
- $11$
- Within a 2D-grid using row-major or column-major order, e.g. as a file manager might lay out files within a directory.
- Row-major; The first 5 prime numbers are:
$2$ $3$ $5$ $7$ $11$ - Column-major; The first 5 prime numbers are:
$2$ $5$ $11$ $3$ $7$
- Row-major; The first 5 prime numbers are:
- Within a 3D grid using analogs of row/column ordering.
- In a circle, assuming that the sequence actually is cyclic, as in this color representation of the 12-tone diatonic scale.
- As a linear DAG, e.g. as in the classic computer science portrayal of a linked list.
- Vertically, going upward, e.g. as a stack.
A set is a collection of elements without any notion of ordering, but with notions of element uniqueness and decidability of element membership. Additionally, a set possesses operations of union, intersection, and difference. Sets can be finite or infinite, countable or uncountable. The abstract data type notion of set is a bit more practical than the mathematical notion of set, but both are valid, structured concepts.
There are particular types of sets which may have additional structure, such as:
Regarding ordering, a particularly important kind is lexicographical order, which allows an ordering to be defined on composite data defined using totally ordered sets.
Examples of sets:
- Songs written by a particular artist -- could be partially ordered based on year of release.
- A list of words or names -- could be totally ordered via alphabetization or partially ordered based on number of letters.
- A subset of numbers, such as the set of prime integers -- could be totally ordered by numerical order.
- The set of emoji within some context -- emoji don't really have a natural (non-contrived) ordering, hence the search feature of the linked site.
- The set of one's living relatives -- could be partially ordered by generation.
- A level set of a function, e.g. contour lines.
- A collection of subsets of a set, having a natural partial order.
Visualization of sets will either involve presenting elements in an arbitrary order (since visual/storage media will necessarily have to apply some order to the elements to represent them), or using any existing structure present in the elements or in the partial/total order to dictate the presentation.
A map is synonymous with function, in that for each element of a domain set it renders an element of a codomain set, but in the context of ADTs, a map generally connotes that the domain will be finite and the element mapping will be explicitly prescribed, instead of formulaically or algorithmically defined. A particularly important special case of a map is that of a bijective map, in which the mapping can be reversed.
In computer science, a map is closely related to or synonymous with an associative array. A typical definition of map/function/associative array specifies the element mapping as key-value pairs, which is the basis for many of its natural visual representations.
Maps can be visualized in different ways. Typical representations are :
- In-line enumeration of key-value pairs, suitable for maps having few elements, for example, mapping a person's name to their favorite color: (Derrex, blue), (Krindy, indigo), (Toresas, red), (Isaberi, yellow)
- Bulletpointed list:
- Derrex : blue
- Krindy : indigo
- Toresas : red
- Isaberi : yellow
- In table form:
Derrex blue Krindy indigo Toresas red Isaberi yellow - As a visualization using arrows to indicate the mapping.
Examples of maps:
- A histogram measuring the frequency of named events. For example, customer feedback about some building contracting service:
Customer was totally satisfied 75 Customer had complaints that were resolved 11 Customer filed civil lawsuit 1 Customer canceled contract early 3 - A lookup table, which stores the result of a computation in order to ease or speed up the computation. Examples:
- Multiplication table (the domain consists of pairs of the numbers being multiplied)
- In dynamic programming, for example in this visualization of the execution of a dynamic program to compute Fibonacci numbers (enter a number in the text box and then click the "Fibonacci table" button.
- Conversion between lowercase and uppercase Latin letters. This is a bijective map.
- Index mapping in a relational database.
- A hash table, which has fast operations for lookup, insert, and delete.
- A switch statement in a computer program, considered as a grammatical construct which analyzes a value and assigns to each possible case of that value what code should be run. The domain of the map is the possible values, and the codomain of the map is code.
A tree structure, synonymous with hierarchy, is a specific kind of directed acyclic graph in which each node may have a number of "child" nodes, and therefore the graph has a "branching" nature to it, hence the name "tree".
Examples of trees and hierarchies:
- Subtype hierarchies, for example:
- Taxonomic hierarchy in the context of biology.
- Class hierarchy in the context of computer programming languages.
- Directory tree structure of a filesystem.
- Authority/organizational hierarchy.
- Level Of Detail (LOD) structured expression, as discussed earlier in this article.
- Abstract syntax tree (AST) in the context of linguistics.
- Binary search tree in the context of computer algorithms.
- The DOM tree of an HTML-based webpage.
- A quadtree or octree in the context of computer games and graphics.
There are many ways to visualize a tree/hierarchy, documented in gorgeous detail at treevis.net (you should just go there), but just to list a few here:
- As a directed acyclic graph, usually displaying each generation of child nodes in mutual visual alignment.
- As a tree mapping, which shows child nodes as literally contained within their parent nodes.
- As nested sets, similar to tree mapping.
- As a tree view, which presents each node in the tree using a UI element which can collapse/expand in order to reasonably navigate the tree, which may have very many nodes.
- As a radial tree, which places a node (typically the root node) at the center, and each "generation" of nodes occupies a place on a corresponding concentric circle centered on the placed node.
A tree may also be represented in static text, using an indentation convention to denote the tree structure. This is a rather unwieldy form however, since it can't be navigated or edited efficiently, as all nodes are visible and are placed in a linear, visual order. Furthermore, child nodes may appear visually distant from their parent nodes. Regardless, here's an example of a subtype hierarchy of ADTs:
- Abstract Data Type
- Sequence
- Tuple
- Array
- Multidimensional array
- Circular array
- List
- Graph
- Directed graph
- Directed acyclic graph
- Tree
- Multitree
- Directed acyclic graph
- Bipartite graph
- Directed graph
- Set
- Ordered set
- Map
- Bijective map
- Sequence
In some sense, the tree/hierarchy is the most common structure in the conceptualization of the physical universe, in that it is the essence of the nested nature of physical objects and their components.
Humans naturally categorize things, and because each category of things naturally admits further categorization based on finer detail, subtype hierarchies naturally emerge from this process.
Both analogy and isomorphism prescribe a correspondence between the respective elements of two collections of things where that correspondence is said to "respect" some relevant structure* present in both of those collections. Said differently, the correspondence
- demonstrates the manifestation of a particular structure, which
- occurs in both collections, each in a way specific to their particular representation.
The structure should be thought of as the abstract essence common to the two collections, whereas each collection is a particular, concrete representation of that structure. Because structure can only be concretely manifest in particular examples, arguably it's only the analogy/isomorphism that really is the structure itself.
This is not to say that the analogy/isomorphism is necessarily total in demonstrating all possible essential commonalities between the two collections of things.
A great visual affordance for faithfully representing an analogy is a table, in that there are two axes of variation:
- One axis for which collection (e.g. animal vs vehicle, as in the following section)
- One axis for which specific element of the collection
A table is a faithful visual affordance for a categorical correspondence as is typically the case for an analogy, but is not in general appropriate for a mathematical isomorphism, as typically isomorphisms deal with non-discrete collections, such as continuous spaces of numbers or vectors or other mathematical spaces. There are a limited number of situations where a table, enumerating a finite number of specific correspondences, is appropriate:
- the isomorphic collections are finite and the correspondence does not have a simple formula, or
- giving a handful of explicit element correspondences is informative, perhaps giving the reader enough information to guess/extrapolate the correspondence without needing to read the formula defining the isomorphism, or
- there are particular special element correspondences which fully define or almost fully define the isomorphism.
Thus it is only recommended to use a table to express isomorphism in those cases. Mathematical objects typically have structures that have very specific visual representations, so it's difficult to prescribe a solution in general. The visual affordance should be selected based on the specifics of the mathematical structures involved.
*Structure, regularity, relationship, or pattern.
For example, one might present an analogy in different forms, each visually representing the essential structure in varying depth of detail:
A bird is to an airplane as a fish is to a submarine.
- A bird is to an airplane as
- a fish is to a submarine.
bird | airplane |
fish | submarine |
animal | vehicle | |
air | bird | airplane |
water | fish | submarine |
nature of thing | $\rightarrow$ | ||
realm | animal | vehicle | |
$\downarrow$ | air | bird | airplane |
water | fish | submarine |
where the analogy demonstrates the essential commonality of:
- realm of existence (air vs water),
- nature of thing (animal vs vehicle),
- means of locomotion (flying vs swimming) -- implicit in decorated tables but could be made explicit,
- origin of thing (natural life form vs human-made) -- implicit in decorated tables but could be made explicit,
but doesn't posit any further structure that might be present, such as kind of energy source (metabolizing food vs burning fuel).
Another example which presents more structure than a conventional "2x2" analogy, and is therefore poorly represented in prose or even as a bulletpointed list:
A prehistoric hunter is to an ancient Greek soldier is to a medieval, European knight is to modern infantryman as a pointed stick is to a bronze-tipped spear is to a steel sword is to an assault rifle as animal hide clothing is to a bronze breastplate is to a plated suit of armor is to kevlar body armor as no shield is to a wooden shield is to a steel shield is to a ballistic shield.
Note how incomprehensible expressing this detailed of an analogy in paragraphical prose is. The "no shield" sounds awkward in this form. Each element of the analogy is visually indistinct, and the relationships aren't clear until you fully read and understand the paragraph.
- A prehistoric hunter is to an ancient Greek soldier is to a medieval, European knight is to modern infantryman as
- a pointed stick is to a bronze-tipped spear is to a steel sword is to an assault rifle as
- animal hide clothing is to a bronze breastplate is to a plated suit of armor is to kevlar body armor as
- no shield is to a wooden shield is to a steel shield is to a ballistic shield.
This is a little better than the paragraphical prose form, in that at least it's clear that there are 4 categories of things in the analogy, but the individual elements are still visually indistinct.
prehistoric hunter | ancient Greek soldier | medieval, European knight | modern infantryman |
pointed stick | bronze-tipped spear | steel sword | assault rifle |
animal hide clothing | bronze breastplate | plated suit of armor | kevlar body armor |
no shield | wooden shield | steel shield | ballistic shield |
prehistoric | ancient history | medieval history | modern day | |
combatant | prehistoric hunter | ancient Greek soldier | medieval, European knight | modern infantryman |
weapon | pointed stick | bronze-tipped spear | steel sword | assault rifle |
body armor | animal hide clothing | bronze breastplate | plated suit of armor | kevlar body armor |
shield | no shield | wooden shield | steel shield | ballistic shield |
time period | $\rightarrow$ | ||||
element of combat | prehistoric | ancient history | medieval history | modern day | |
$\downarrow$ | combatant | prehistoric hunter | ancient Greek soldier | medieval, European knight | modern infantryman |
weapon | pointed stick | bronze-tipped spear | steel sword | assault rifle | |
body armor | animal hide clothing | bronze breastplate | plated suit of armor | kevlar body armor | |
shield | no shield | wooden shield | steel shield | ballistic shield |
The examples given here will be presented in table form as described above, for explicitness of correspondence*. An isomorphism typically describes what mathematical structure it is preserving, and may also include a formula which defines the correspondence between the two collections of things. However, as described in a previous section, a table is only a good visual affordance for isomorphism in limited cases.
*A tabular, element-by-element mapping is typically not ideal in the mathematical setting unless the sets are finite and a formula is not available or is convoluted. An explicit element-by-element mapping can serve as a lookup; a computational convenience.
$\mathbb{Z}$ (ring of integers) |
$n \mapsto 3n$ | $3\mathbb{Z}$ (ring of integer multiples of 3) |
... | ... | |
$-2$ | $-6$ | |
$-1$ | $-3$ | |
$0$ | $0$ | |
$1$ | $3$ | |
$2$ | $6$ | |
... | ... |
$\mathbb{R}+2 \pi \mathbb{Z}$ (group (addition) of equivalent angles) |
$\theta+2\pi \mathbb{Z} \mapsto (\cos(\theta), \sin(\theta))$ | $\mathbb{S}^{1} \subset \mathbb{C}$ (group (multiplication) of unit complex numbers) |
... | ... | |
$0+2\pi \mathbb{Z}$ | $1$ | |
$\pi/6+2\pi \mathbb{Z}$ | $\sqrt{3}/2+i/2$ | |
$\pi/4+2\pi \mathbb{Z}$ | $\sqrt{2}/2+i\sqrt{2}/2$ | |
$\pi/2+2\pi \mathbb{Z}$ | $i$ | |
$\pi+2\pi \mathbb{Z}$ | $-1$ | |
... | ... |
Here is a beautiful example of an isomorphism between partially ordered sets, where the structures involved are divisibility of integers on the left and set inclusion on the right.
There are obvious ways that structured concepts have spatial aspects, such as any numerical or geometric concepts. However, there are non-obvious ways that structured concepts can naturally translate into a spatial representation. Here are a few examples of how structured expression manifests spatially:
- Prototypical time series: A sequence can be interpreted as a map/function from ${0, 1, 2, \dots}$ (prototypical time values)* to the numerical values (prototypical measurement values) in order to produce a plot. There are numerous ways to convey additional information about the function's domain and/or codomain, including:
- As discrete points -- no information about continuity conveyed. See also scatter plot.
- As piecewise-constant segments -- hints that the domain is continuous (e.g. time). See also bar chart.
- As a piecewise-linear curve -- hints that both the domain and codomain are continuous, but does not assert any higher order smoothness to the function. See also line graph.
- As an interpolated curve -- hints that the sequence is a discrete sampling of a smooth function, but it should be noted that any such interpolation, absent additional information about the nature of the function, only gives an approximation of the values missing from the sequence.
- Prototypical parameterized path -- 2D or 3D: Given 2 or 3 equal-length sequences, then they can be interpreted as coordinate functions for a curve in 2-space or 3-space respectively. As with the time series, different information about the domain/codomain can be conveyed using different plot types. Examples:
- As discrete points (the red points) -- no continuity information.
- As a piecewise-linear curve -- function is continuous, but no smoothness implied.
- As an interpolated curve (the blue curve) -- function is visualized using a smooth approximation.
- Translating ordering into direction -- some other structures have some notion of an ordering, which can be encoded visually by mapping it onto a spatial axis.
- A tree structure -- the order on the tree nodes is given by the successive generations. Each generation of nodes can be visually aligned to indicate the generational structure. If the branching factor for the tree is too great, then this visual affordance may not be feasible and perhaps interactivity would be necessary.
- or a directed acyclic graph (DAG) -- as in the case of the tree structure, there is a notion of generation and therefore of visual alignment to indicate generation, though each node may have multiple ancestors. Again, as with the tree structure, if the graph is too dense, then an interactive affordance may be needed.
- Distance and difference -- spatial distance between visual elements can be used to indicate distance/difference in the structure.
- A tree structure can show distance from the root node via sunburst diagram.
- A graph's connectivity/sparsity can be visualized using a force directed graph.
Again, The Data Visualisation Catalogue is an excellent resource, presenting a categorization of visualization technique based on function.
*Use of ${0, 1, 2, \dots}$ as the index set is a natural convention for computerized contexts based on how the elements of an array are stored. In non-computerized contexts, humans' natural counting convention is used, where the index set is ${1, 2, 3, \dots}$.
Instead of a monolithic block of visually indistinct text, the addition of these organizational structures allow for basic, efficient topic-based organization and navigation of written content. A Table Of Contents could be considered to be a primitive, non-interactive form of Level Of Detail.
The maxim "a picture is worth a thousand words" perhaps doesn't go far enough. Some visualizations really have no faithful rendering into prose, and by convention would fall to using metaphorical language to convey some semblance of the vision to the human reader. Exceeding the limitation of prose is the crux of the Structured Expression Project Thesis.
Grammatical or semantic content is indicated by the style or the color of text. Italics is often used to indicate emphasis or use of a foreign word. Boldface may indicate introduction of a new term. Use of a different font may indicate a quotation or content belonging to a separate context. Syntax highlighting is commonly used to make computer program source code more readable, especially regarding comments and numeric/string literals.
Scientific language is a kind of sublanguage which is an invaluable tool, adapting natural languages such that reasonably accurate scientific inquiry and reasoning is possible. This allows natural languages to function on a literal level, away from the vagueness of metaphorical interpretation, and in particular allows for concrete verbal reasoning about an objective reality.
modern style which intersperses equations, graphs, visualizations, and symbolic affordances
Take for example an exerpt from Newton's Principia (Book I, `Axioms, Or Laws Of Motion`; page 85 in print, which is page 90 of the PDF):
There are no pictures or visual affordances given. This requires the reader to do tremendous cognitive legwork in visualizing the physicality of the description. Isaac Newton was an intellect nearly unrivaled in all of human history, yet the stilted writing style with its adherence to prose produced an awkward, grammatically incomprehensible mess. Yet the concept could have been conveyed directly and clearly in a diagram, taking advantage of humans' significant visual and physical reasoning abilities.
Newton's exposition does have a number of drawings, nearly all of which take the style of geometric proof, with constructions of lines and arcs, labeled points and segments, etc.
Contrast this with the more modern style which intersperses equations, graphs, visualizations, and symbolic affordances. The visualizations deftly take advantage of humans' powerful visual processing faculties, making certain concepts obvious. While symbolic expressions do require the reader to become familiar with the symbolic "vocabulary", they also lend themselves to certain visual regularities that the reader can gradually gain an intuition for quickly understanding them on a cursory level.
These place information which is related to the content in a nearby location in order to avoid interrupting the linear flow of the main written prose. This is perhaps the simplest example of nonlinear structure in the presented content being rendered in a correspondingly nonlinear way -- in other words, not as a dependent clause or parenthetical within the sentence.
The ability to navigate through bodies of writing in a nonlinear manner based on the reader's choice is the essential concept here. References and citations, by which I mean textual references within a book to either another section of the book, or another book entirely, could be considered to be the hand-powered predecessor to hypertext, which necessarily involves an interactive interface, such as a web browser, to navigate to the referent.
The ability to tag content with particular names, indicating an association of that name to the content, thereby facilitating the creation of an association index. This is an example of a distinct, overlapping structure in addition to whatever structure is already present in the content being tagged. Examples:
- Stackoverflow content tagged with polymorphism
- Arxiv content tagged for Differential Geometry: math.DG
- The index of a book, in the conventional style of publication
Examples will be given directly in this section. Unfortunately, the anchor links below which refer to other parts of this article don't work unless that section is already expanded, so for now, I would recommend expanding the whole article before clicking any of those links.*
Expand the whole article Collapse the whole article
Examples of structured expression as applied the humanities:
- Level Of Detail is a structured expression that is useful for practically any exposition, regardless of its topic.
- Parallel channels of exposition is also a perennially useful structure. Particular examples in the humanities:
- Side-by-side correspondence of a text and its translation(s), showing the correspondence between each sentence.
- A body of text with a sidebar giving context/commentary without interrupting the flow of the main text.
- A screenplay, with columns for speaker, dialog, and associated emotion.
- Examples for biography, novel, story, narrative.
- A flow chart, for example representing the overall plot structure of a story and its character interactions.
- Complex, multi-part analogies can be practical with the appropriate visual affordances.
- Lists rendered using bulletpoints to attract the eye and make the existence of a list and the separateness of its elements visually obvious.
- A rhetorical argument has a directed acyclic graph structure which shows which statements support which other statements, allowing the logical flow of the argument to be clearly, visually represented.
- Using succinct formatting to encode semantic information which would otherwise require superfluous grammatical padding.
Finally, this site is an index into many forms of text visualization, each of which likely make use of much structured expression.
*This is an example of one of the many ways that I find trying to use existing web content tools for structured expression totally unsatisfactory.
I'd like to thank the following people for the many conversations that contributed to the ideas in this article. It's been a total delight to experience the feeling of my mind expanding during these conversations. In no particular order*:
- Raffi Bedikian
- Max Sills
- Lauren Peritz
- Seth Heltsley
- Daniel Palm
- Marissa Simons
- Ben LaCara
- Nick LaCara
- Ellie Wytychak
- David DeConde
I would also like to thank Wikipedia, the Wikimedia Foundation, and the countless Wikipedia contributors. It really can't be understated how revolutionary Wikipedia is**. I donate monthly to the Wikimedia Foundation because I think it's one of the most important resources on the internet, and I would suggest that if you value it, that you consider donating as well. Wikipedia is one of few instances showing that indeed, We Can Have Nice Things.
*A structured expression which would be useful here would be an unordered set, in which the elements (names, in this case) are presented in a random order, perhaps even distributed randomly over a 2D region on the screen. This would be a truly faithful manifestation of "in no particular order".
**Wikipedia is an unmitigated success even just in terms of access to such a breadth of knowledge. But consider also that it's created by its users (you and me), that it's a free resource, and is intentionally untrammeled by the advertisement economy that has changed most of the internet into the attention-hungry, data-mining blight it is today.
I'll soon be making a page for the Structured Expression Project, which aims to create various software tools (and develop practical conventions) for use of structured expression.
There are a number of improvements I would like to make to this article and the tools used to create this article. In particular:
- Add mouse-over for links that shows a small pop-up iframe for certain links, just like Wikipedia does for its internal links. This would give a more seamless experience in that one could preview linked content without leaving the article page.
- Add a way to expand all LOD elements, so that the article's content can be searched.
- Probably put more of the detailed content into LOD form.
- Add pictures and visualizations where useful. The tools to do so aren't easily accessible in the medium of this article (WordPress with some plugins), so I've relied heavily on external links to visual content.