On Making up Words (up to p126)

And so we have more Exposition behind the inner workings and reasoning of the Jumbo program. The reason for the title in this case is due to the fact that by the end of the chapter he has introduced an unusual amount of made up terms in the chapter, starting with many utensil based variations of the term spoonerism (something that I have been aware of and very susceptible to for a long time, though I never knew it had a name), and then going to pseudo mathematical concepts like "thinkodynamics". Considering the nature of the program to generate spontaneous english sounding words, this seems more than coincidental. The creation of terminology isn't the issue here though.

I'll be honest, I've never really done jumbles. It's not that I feel I'm not capable of it, it just never really interested me. However, I have played games of scrabble (The competition can be pretty fierce in my house), and the concept of Jumbo almost seems to have a closer application there then it does to jumbles. The main difference is that you know there is an English word in the jumble that uses exactly all the letters. In Scrabble, there probably is one, but it probably doesn't use all the letters, and also has the added complication of having exactly one letter that's already been played on the board. Never mind spacing issues and point values and whatnot.

One sort of thing that isn't included in the program that I tend to do very regularly in scrabble is come up with a combination that is a letter short. Much of it is just wishful thinking, but it's also a very integral part to playing the game, as you can't possible keep track of all the letters that are on the board at a given time, so there is some sort of imaginary blank tile that's always in your set. This seems different, because instead of just dealing with the existing letters, you are doing the same tossing up into the air, but somewhere in the process you have added letters that are not in the set. Toss it up again and a different letter may come down. With those instances where you actually have one or both of the blank tiles, you can be tossing up and coming down with up to 3 completely random letters at any given time. The reason why the methods Hofstadter describes seem most ideal in this situation is the fact that you can't rely purely on your own vocabulary when playing scrabble. There is invariably a dictionary on hand for when what looks like a possible word needs to be double checked. Of course, the word forming ability of pro scrabble players probably forms words that look nothing like english on a fairly regular basis, as letter combinations rarely seen in english (such as words that use a Q without a U after it) are absolutely essential to come out ahead in the game.

Nonetheless, the cognitive model that Jumbo relies on seems like it would apply to scrabble very well. In fact, only little modification would be necessary to make it cover scrabble ideally. But the fact that it is possible to fill in letter combinations that are a few letters short is rather remarkable. The gloms that are so essential to the jumbo system are probably key, but where the extra letters come from is the mystery. Could be completely random, but more likely it's a comparison to the glom rules that the letters come together with, and just picking a letter that seems most ideal. Of course, that letter would still have to be distinguishable from the actual letters, as the breaking down and reforming process that is discussed would still need the option of changing the letter without completely breaking down the word. Of course, I could be completely wrong, but that's how I think it might work.

More on Glomming (up to p111).

This particular reading was mostly the same topic as the last one, that of his concept of Glomming. The concept seems very much like an extension of the ideas of islands mentioned about Seek Whence. Instead of having numerical runs of different sorts, it's a combination of letters that fit together logically. The Gloms are just combinations of islands like before. The main addition to this system other than the retooling for language is a way of overcoming some of the ambiguity of it, by weighting the different combinations. The most interesting part of the program mentioned so far is the use of randomness. Usually randomness is to be avoided in programming, as its non-systematic nature tends to actually make things longer. And yet, the analogy he uses makes sense, that putting together chemicals in a cell is a mostly random process, but still results in the necessary chemicals. It feels like a fairly effective technique at modeling how we can unscramble words in our minds, with one exception: We do have a dictionary in us, full of thousands of words. While we definitely make combinations in a way similar if not identical to the technique the program relies on, I think we are also constantly matching partial words, and discarding many gloms before the word has fully formed because of that. It works as a synthesis of the patterns found in English, but as a total map of the process used in unscrambling a word it seems to fall short. Of course, he probably has more ideas that will account for that, and it is a very interesting idea whatever the case.

Citrus Viewer Ring (up to p95)

Anagrams can be fun, funny, and also the main topic of a reading. Your mileage may vary on the first two, but in this situation at least, the third is definitely true. All of the talk about letters glomming automatically in our minds and things like creating a particular set of letter objects out of the letter objects in the alphabet brought me hard to thinking about the process of learning a language in the first place. After all, unlike speech, the written word isn't something that is acquired merely through exposure, it requires a conscious effort and regular practice until it becomes a fairly autonomous process. For most of us, we learned to read so long ago that we can hardly remember any of the difficulties associated with it, but it most definitely took a matter of years before the letters on a page started to automatically glom together to form coherent words, instead of having to think and sound out what might be the word (english is notoriously bad for this, being basically a hodgepodge of spelling rules and pronunciations from across most of europe and a little bit beyond). Of course, if someone wants to remember what this is like, they merely have to take a language that has little relation to their native one.

This semester I have had the opportunity to take the first class in Japanese. Unlike learning most western european languages which all use an alphabet that is fairly similar to english and also have many very similar if not identical words, japanese is very different. For starters, the bulk of the language is ideographic, but it can still be represented through the use of the other two phonetic "alphabets" of Katakana and Hiragana. Both of these represent the same sounds, but those sounds are not the same as english. There are nearly 50 characters in each, and none of the characters have more than a passing resemblance to an english letter. This is besides the fact that none of the words have origins near english, with the exception of loan words, which are usually pronounced and sometimes even used in a drastically different manner than their origin. Calling it difficult feels like an understatement. Having learned Katakana already, I am to the point that I can pronounce things written in that language, but it is a slow process of culling my memory for what that character sounds like and stringing them together to form a word. Fortunately, Japanese has few exceptions in its pronunciation, and once you learn what the phonetic characters are, you can pronounce it fairly accurately despite not being familiar with the word or what it means. No need for the advanced glomming techniques that he mentions in the reading, only the need to recognize that it is a word.

So was learning to read english this hard at first? I can't say myself, I only remember little things from the process: flash cards with letters and words on them, having teachers regularly explaining the difference between b and d, writing line upon line of single letters for practice. Certainly, the process sounds the same. So that brings up a small developmental question of whether or not we really do learn faster as children. So much less knowledge to build from, and to form analogies with. It would seem that it wasn't easier, just that the difficulty of it is so easily lost in the mists of human memory by the time researchers start to really think about it. Whatever the case is, I don't think I'll be attempting foreign language anagrams anytime soon.

It's like an Orange, but Deeper (up to p86)

Despite the great ease that this final section of the chapter on the Seek-Whence program has of allowing a music related rant, I will try to keep that to a minimum, as it doesn't really seem pertinent as anything more than a useful analogy for the other concepts being discussed. The concept of variations on a theme is in this case being loaned to the idea of taking an existing sequence or more general concept and varying on it. The main question posed in this section is essentially how far can you diverge from a given concept before the relationship is too tenuous to really be valid. I would instinctually feel that how far of a stretch something is would at least be partially relative to the individual.

Take for instance, the title of this post. This came about from a conversation with a friend of mine about iambic pentameter. At which point he made the aforementioned analogy: Iambic pentameter is like an orange, but deeper. Now, he and I both agreed that it was a hilariously ludicrous application of simile, but somewhere in the back of his mind there was enough of a connection for that idea to come up. It may have been on the fringes of the halo, but sure enough, it was close enough to count.

It illustrates the concept well enough, just like the examples about things such as spiked tylenol. Certainly, the same concept can be applied to other things, but once the action is changed in form, it was too much of a stretch to see directly. After all, if he had said that spiking tylenol was like throwing money out of a helicopter, only bad, I would think we had another Deeper Than An Orange case. Of course, the connection was made more clear by the use of "spiking". Indeed, the quotation marks have the power to make tenuous relationships that much closer, which is a funny example of the me-too effect mentioned also in this reading. All of the sudden, something that wouldn't make much sense jumps into reasonableness.

The music term Variations on a Theme is highly appropriate for this sort of behavior, as to make a valid variation, certain things have to stay the same, mainly being the general melody and rhythm in the larger sense, as well as the Chords. This of course still leaves room for extra non chord tones, chord extensions, key changes, mode changes, time signature changes, etc. But if you did things like invert and reverse the melody, or change the large scale harmonic structure, it would no longer be a valid variation and would quickly go into the realm of unrecognizable. And so it would seem with the scenarios he mentioned in the book. Two major items that can't be varied would be the higher order classification of the subject (you still inspect bridges instead of tunnels, or roads), and by extension, the action (due to the more strict classification of an action). Thus, anything that doesn't involve poisoning some form of consumable item just doesn't have the same spirit about it, even if it is an equally malicious act. Of course, perhaps my definition is too conservative, and the spirit in which the act is done is more important (entirely anonymous and random) to the variation. Who knows.

Mountain Building (up to p70)

I will try to avoid any further outbursts on the subject of music that veer away from the core of the subject matter too much in the future, and thankfully the subject matter will probably facilitate this. This section of reading focused around a specific pattern known as a mountain chain sequence. This sequence ultimately has a fairly straight forward pattern, but the method of devising it introduced in this section is new, and probably closer to what a human being does. In short, it takes a small chunk of data, groups it into sets of related data (runs, plateaus, etc.) tries to find a reasonable connection between these different sets to form a hypothesis, and then adds in an adjacent chunk of data, and then goes back and fixes the hypothesis if it doesn't quite work. From personal experience trying to solve pattern sequences and other similar math puzzles, this makes sense.

The Crypto problems from this class is a good example of this. Certainly, 5 numbers and 4 operations isn't exactly something that is going to have multiple levels of relationships like the mountain chain sequence, and it is also something that is an ideal candidate for brute force methods. But we humans, due to the raw math processing power we lack to be able to do something like a brute force search, we tend to look for a fairly obvious relationship between numbers and work from there. After a first result, we add that number to the set of remaining numbers, and repeat the process until we run out. If it doesn't work, we go back a couple steps and rework the numbers until we get the answer. Of course, this probably isn't too suprising to anyone who successfully solves these problems. What is suprising, however, is the difference between the solutions that members of the class came up with. To me, the operations have a precedence that I tend to look for: Subtraction, addition, multiplication and then division. As such, my solutions tend to be almost entirely subtraction and addition. This seems fairly reasonable to me, but then came a series of entirely different answers to the same problems. It would seem that despite what should be a fairly consistent educational grooming throughout our childhoods, there seems to be a good deal of room for us to diverge. Whether or not that divergence is due to nature or nurture is a good question, but not really relevant to the current topic. Still, I would put my money on nature.

The meeting of my majors (up to p55)

When telling people that I major in Computer Science and Music, that tends to lead to some questions as to how those two work together. Admittedly, the connection between them exists mostly in a deep, theoretical sense, as both do have certain mathematical roots, but rarely do the standard planes of their studies actually intersect. So for it to show up in this reading was as much a delight as it was troublesome.

How music comes about from the previous discussion of sequences is due to the fact that the mathematical sequences usually require a good deal of background knowledge in math for the patterns to be recognized. Believing the two ideas to be unrelated, he mentions there being a Math portion and a Pattern portion to recognition. Indeed, showing kindergartners these sorts of mathematical puzzles (even the relatively simple ones) and they probably would not be able to tell you too much about them, the math knowledge needed (basically anything beyond counting at that point) is just not there. On the other hand, if you show them a sequence of shapes or colors or something a little less formal and directly knowledge intensive, and they can probably pick out a pattern, possibly even some fairly elaborate ones. The pattern is there, the math is not. Which is why music pops up. I can assert from first hand experience, that despite the correlation between Computer Scientists and a strong interest in music, the vast majority of music majors and minors are about as interested in math on average that anyone else in an art field would be. It's not that the math is not there, it's just not necessary for the creation or enjoyment of it.

And thus, that is why the crossing of the subjects here is troubling. When I first started studying music, I was convinced that there was basically a magic formula that all proper music followed. By the time I was done taking the college theory courses, it turned out that theory was merely the observations of people about what composers have done over the course of the past few hundred years. If you go and ask most composers about why they wrote something in a particular way, the main answer (even if they try to make it some more sophisticated and intelligent than this) is ultimately because it sounds good. Now, the reason why it sounds good is the interplay between the very tidy mathematical ratios found in the harmonic series, but that's not the point here. The patterns are capable of being analyzed without a need of mathematical background or even music theory (since it was created after music anyways). But taking it out of the context of the music as just a chain of letters or whatnot kind of kills it. Most musicians could see the sequence on a staff and immediately get what's going on, no ambiguity between the contrapuntal lines at all. Indeed, what he mentions on the very last page of this section of reading probably states the whole thing best. Merely being able to recognize a pattern in music in this way does not in any way state the music. It has become just a series of numbers, no more music than a DNA sequence on a computer is the person to whom it belongs. It is an inherently human creation, full of the complex emotions and nuances of their creators. My beloved Music Theory is not perfect in this respect either, but it is at least designed to properly reflect music, and is therefore more human than math. If I were to design a system to analyze the patterns in music, it would be done through the use of music theory, as the patterns would require just as little background to recognize, but would not completely strip it of its humanity.

Intro and first Reading (to p35)

Well, here is my blog. I had no intention of ever making such a thing as this, considering that my opinions on matters are relevant to me, and most likely no one else. But as required, here it is.

The first reading, in short, recounts the tale of finding a pattern between triangle and square numbers when interleafed by their appropriate quantity when he was sixteen. The mathematical definitions he used for triangle and square numbers are interesting, as their pattern is actually very similar to two of the basic sound waves, the saw wave (all multiples of the base frequencies), and the square wave (only odd multiples of the frequency). The actual Triangle wave, however, is actually closer to the square wave in harmonic content. Anyways, the pattern that he eventually discovered was a recursive pattern between the two interleaved sets. The emphasis he had on the issue of using ellipses to show the continuance of the patterns comes to light with this, as it only works when the base pattern is so apparent that someone can extrapolate the result. Something that recurses up infinite levels like the pattern described cannot be used with such a pattern. Of course, the use of grammars and the like make describing something like that much less ambiguous.

The main subject of this reading, however, was getting a computer to be capable of solving a large range of number patterns. The issue is not so much getting the computer to do different operations with the given data, but to recognize the pattern. This manages to highlight well the difference between human behavior and machines, as you can easily get a machine to do a large amount of operations before even coming to a relevant one, whereas a human can just choose an almost abitrary point in the middle of a pattern and see a relationship with the group of numbers. Not being psych or even a cog major myself, I know almost in passing about the semantic network model, where certain concepts immediately lead to (at least according to the person) the next closest related concepts. This may be the source of human intuition in these matters, as somewhere in our mind various mathematical concepts will stand out more than others. The computer methods of Breadth first and Depth first are as far from intuitive as you can get. If a human being decided to just up and try something like that to find a pattern, they would most likely be chipping away at it for an exponentially larger period of time than someone who just guessed and checked. Even his concept of sniffing, while alleviating some of this, still has some issue, as it still basically goes through a laundry list of mathematical concepts, just not to the large degree that the actual sources do. And yet while the human process seems entirely random, it is not. After all, a basic programming exercise is to guess a number by knowing only higher or lower. This is done by starting in the middle and moving to the midpoint of the new set, not by using a random number generating and hope that the dice roll lands on the right number. Of course, a human being is capable of guessing on the first try sometimes. Not because of luck or through some algorithm, but through that mysterious force of intuition. After all, just like the guesser, the person thinking of the number is not capable of truly generating a random number.

But I digress. If the semantic network idea holds any water, it would seem that the realm of intution in algorithm form would lie somewhere between the two extremes of breadth first and depth first. Of course, since the two searches are related to trees and not a network on the same sense as the model mentioned before, it is probably very hard to bridge the gap and come up with an appropriate algorithm based on this. I am sure, however, that Hofstadter has more of his personal insights on the matter to reveal.