The idea of the conceptual spheres is by far my favorite that I have discovered in the course of reading this book so far. It has a very visual appeal to it, immediately conjuring an image of something akin to a field of stars, with coronas designating if a given idea has a close enough relationship. This new concept of slippage is so closely related to it in my mind that is merely a minor extension of it, representing the pull these coronas have on each other and other concepts, and the ambiguity of categorization. The idea of slippage had already been in my mind from previous thoughts concerning the categorization of music into genres. Genre is something that is incredibly complex from a historical perspective, yet practically amorphous from a current day perspective. What constitutes a genre can be almost anything, the particular instruments used, the particular rhythms, subject matter and/or origin of the lyrics, etc. A more simplified definition of genre would probably be along the lines of how closely a song falls to a given Ur-example.
The Ur-example historically for rap would be Rapper's Delight, from 1979. However, since the music industry is driven primarily by teenagers, none of them would be old enough to remember this, so something more recently would be the current Ur-example that a new song is compared against (or to be more accurate, made to fit). So time goes by, and the old standard is well outside of the new standard's halo. Obviously, the old standard did not cease to belong to the genre all of the sudden, so then subgenres are born around different standards. But then you have pieces that are clearly in both halos. So which sub-genre would that be in? In musicology it is usually considered a transition period, but at the same time, someone might feel like looking back and making music that reflects the older style, so is that now categorically in the older style, but since that's no longer what it's being measured against, is it something else? And then you have the examples drifting closer to other genres. What would you call Linkin Park? Calling it rap by any name wouldn't seem right, but the similarity cannot be ignored.
And so the Slippage occurs, and the halos duke it out. This probably is in the thoughts of the people creating the music as well. When a recording artist lists their influences, it can be very diverse, if not contradictory. For me, at least, composing a piece of music is much like playing darts. You have something in mind stylistically that you are aiming for, but the end result is never exactly what you planned on. There are too many influences and factors to get something that is right next to what you were aiming for, but should not be too far away. Of course, this too will lead to things that fall between halos almost by accident, and then more slippage, and more genres and sub genres are required. At this point, I try to keep it simple and keep the genres to music I like and music I don't like. But I think I need to listen to some retro-acid-pop-electro-funk.
Wednesday, October 28, 2009
Hindsight, fast track to discovery (through Ch4)
Hindsight is a funny thing, the way it manages to simplify all of life's happenings. I am fairly certain that there is not a person on earth who has made a choice that when looking back on it later from a more objective and well informed position will wonder why they ever made such a decision. Though since the knowledge gained through said situation is usually what allows the hindsight to be effective, it raises the question of how anything ever actually gets accomplished in the first place.
The example of Kepler is a good one, considering all of the many ideas he worked with before the realization of the laws that now bear his name. It is a hard thing to model, to say the least, since the various influences came from concepts that often had little to do with the actual movement of celestial bodies in the sky. Certainly, as mentioned in the reading, finding the formula when given the appropriate table of values is only a little bit above trivial in difficulty. However, when taking into account the difficulty of even finding those values in the first place, then an estimate in the range of years, if not decades would not seem unreasonable before even factoring in the issue of whether or not the data is even pertinent. And such it is with Hindsight: when you already know exactly what you're looking for, it gets much easier to find.
There are a few scientific traps similar to this issue with hindsight. According to a study, people who brush their teeth regularly live an average of two years longer than those that don't. Personally, I would be embarrassed to even attach my name to such a study. We all know that Correlation is not Causality. Brushing your teeth must increase your lifespan, unless people brush their teeth more often as they get older. The more reasonable conclusion would be that the person has a wealth of good habits that tooth-brushing just happens to fall under. Whatever the case, a 2 year increase (in average, no less) isn't exactly worth mentioning. The amount of studies like this have created a modern day superstition about what particular activities or foods will increase your lifespan or what have you, when in actuality, the correlation is tenuously related at best.
The much closer pitfall (a generalization, mostly) of the idea of hindsight is one familiar to most people in the science and engineering fields: Testing. In computer science, a good deal of preliminary testing is usually done by the designer as part of the process, much like handing Kepler a table of values. The standardized tests will always give standardized results. Same goes for placing weights on model bridges and crash testing cars. Eventually it gets deployed to the field, and something unexpected gets thrown at it. A user tries to hit two buttons simultaneously, and the program crashes consistently. A particularly windy day hits the bridge's resonant frequency and causes it to collapse, or a tight turn causes the car to easily roll over. In hindsight, these would all seem like easily testable and preventable issues, but are often overlooked by the standardized testing. In terms of this, the problems with many of the AI programs mentioned in the reading seem quite reasonable. It isn't Hindsight that's the issue so much as Foresight, and that's a much more difficult issue.
The example of Kepler is a good one, considering all of the many ideas he worked with before the realization of the laws that now bear his name. It is a hard thing to model, to say the least, since the various influences came from concepts that often had little to do with the actual movement of celestial bodies in the sky. Certainly, as mentioned in the reading, finding the formula when given the appropriate table of values is only a little bit above trivial in difficulty. However, when taking into account the difficulty of even finding those values in the first place, then an estimate in the range of years, if not decades would not seem unreasonable before even factoring in the issue of whether or not the data is even pertinent. And such it is with Hindsight: when you already know exactly what you're looking for, it gets much easier to find.
There are a few scientific traps similar to this issue with hindsight. According to a study, people who brush their teeth regularly live an average of two years longer than those that don't. Personally, I would be embarrassed to even attach my name to such a study. We all know that Correlation is not Causality. Brushing your teeth must increase your lifespan, unless people brush their teeth more often as they get older. The more reasonable conclusion would be that the person has a wealth of good habits that tooth-brushing just happens to fall under. Whatever the case, a 2 year increase (in average, no less) isn't exactly worth mentioning. The amount of studies like this have created a modern day superstition about what particular activities or foods will increase your lifespan or what have you, when in actuality, the correlation is tenuously related at best.
The much closer pitfall (a generalization, mostly) of the idea of hindsight is one familiar to most people in the science and engineering fields: Testing. In computer science, a good deal of preliminary testing is usually done by the designer as part of the process, much like handing Kepler a table of values. The standardized tests will always give standardized results. Same goes for placing weights on model bridges and crash testing cars. Eventually it gets deployed to the field, and something unexpected gets thrown at it. A user tries to hit two buttons simultaneously, and the program crashes consistently. A particularly windy day hits the bridge's resonant frequency and causes it to collapse, or a tight turn causes the car to easily roll over. In hindsight, these would all seem like easily testable and preventable issues, but are often overlooked by the standardized testing. In terms of this, the problems with many of the AI programs mentioned in the reading seem quite reasonable. It isn't Hindsight that's the issue so much as Foresight, and that's a much more difficult issue.
Sunday, October 25, 2009
My Blog Hates Me (up through pre 4)
Ever since I heard about it a few years ago, I have always been amused by the Eliza Effect. It seems natural, if not inherent, for human beings to anthropomorphize inanimate objects. After all, there isn't a person who hasn't complained about their car or computer or what have you as having a mind of its own and somehow deciding to act up, instead of it being some sort of failure or error inside of the device in question. Combine this with an unexpected ability to make a logical, albeit limited, response to a query or input stimulus, and then people can be pretty confused.
I think it has to be unexpected for it to work. The Eliza effect originated during a generation when computers were not commonplace, and their capabilities were not in the forefront of people's minds, including the people who work with them. Children nowadays have an array of toys that react to them. Games have AI with ever increasing sophistication. There are even a number of bots out there that people can talk to.
Most anyone who has AIM has talked to SmarterChild at least once. Having seen some of the excerpts of conversations people have had with it, it is obvious that the eliza effect has failed. Certainly, there is no accounting for taste on the internet, and anonymity can make some people get offensive much more quickly than they would face to face. All that taken into account, the conversations go beyond that. It is a neat piece of software, but nobody is buying the illusion anymore. Perhaps it's because a casual conversation is too open ended a domain for it to be convincing, unlike the original Eliza's psychologist conversations. Or perhaps people nowadays are rather desensitized to the concept of a talking computer, and will need a much more elaborate technique to be convincing.
His extension outside the realm of mere conversation is much the same. Reading about some of the other supposedly huge accomplishments in the field of AI sounds impressive when given in vague terms when no understanding of how its done is supplied, but usually fall apart when really scrutinized. Writing a method that's supposed to generate a random number but merely returns a 4 is rather convincing the first couple of tries, but becomes glaringly obvious very quickly. Hearing about a novel that was written in a particular style that required a huge amount of questions to be asked to the user was fairly disappointing, and downright embarassing to hear that it was published. I'm sure that it was a program into which a huge amount of work went, but to hear it put as a computer program that writes books in a particular author's style makes it sound like all you have to do is give a few character's names and perhaps a setting, and push the big Go button and you have a novel hundreds of pages long.
The program that understands analogies also is equally lackluster. But then again, to actually recognize things requires a great deal of background knowledge, so it is understandable. Having to spoon feed your program the necessary information about something in order for it to know enough to make a connection is going to fall into this pit by default. Now, if it already had built into it a human being's knowledgebase of predicate statements and could understand an analogy from the midst of that, it would be impressive. Chances are, that it would produce nonsense, and would not have been a noteworthy accomplishment. Really they're all just kids doing a somersault, but the trappings of some are more impressive than others. You're going to pay attention to the one with the laserlights and the pyrotechnics, but if you can see past all of that, you'll be asking "that's it?"
I think it has to be unexpected for it to work. The Eliza effect originated during a generation when computers were not commonplace, and their capabilities were not in the forefront of people's minds, including the people who work with them. Children nowadays have an array of toys that react to them. Games have AI with ever increasing sophistication. There are even a number of bots out there that people can talk to.
Most anyone who has AIM has talked to SmarterChild at least once. Having seen some of the excerpts of conversations people have had with it, it is obvious that the eliza effect has failed. Certainly, there is no accounting for taste on the internet, and anonymity can make some people get offensive much more quickly than they would face to face. All that taken into account, the conversations go beyond that. It is a neat piece of software, but nobody is buying the illusion anymore. Perhaps it's because a casual conversation is too open ended a domain for it to be convincing, unlike the original Eliza's psychologist conversations. Or perhaps people nowadays are rather desensitized to the concept of a talking computer, and will need a much more elaborate technique to be convincing.
His extension outside the realm of mere conversation is much the same. Reading about some of the other supposedly huge accomplishments in the field of AI sounds impressive when given in vague terms when no understanding of how its done is supplied, but usually fall apart when really scrutinized. Writing a method that's supposed to generate a random number but merely returns a 4 is rather convincing the first couple of tries, but becomes glaringly obvious very quickly. Hearing about a novel that was written in a particular style that required a huge amount of questions to be asked to the user was fairly disappointing, and downright embarassing to hear that it was published. I'm sure that it was a program into which a huge amount of work went, but to hear it put as a computer program that writes books in a particular author's style makes it sound like all you have to do is give a few character's names and perhaps a setting, and push the big Go button and you have a novel hundreds of pages long.
The program that understands analogies also is equally lackluster. But then again, to actually recognize things requires a great deal of background knowledge, so it is understandable. Having to spoon feed your program the necessary information about something in order for it to know enough to make a connection is going to fall into this pit by default. Now, if it already had built into it a human being's knowledgebase of predicate statements and could understand an analogy from the midst of that, it would be impressive. Chances are, that it would produce nonsense, and would not have been a noteworthy accomplishment. Really they're all just kids doing a somersault, but the trappings of some are more impressive than others. You're going to pay attention to the one with the laserlights and the pyrotechnics, but if you can see past all of that, you'll be asking "that's it?"
Wednesday, October 7, 2009
Under the Hood (up to p154)
While explanations behind the theory of the previously mentioned programs have made up a good deal of the book so far, none had actually had the step-wise breakdown of the program's operations on an actual example. This reading actually contained a couple, with some other examples mentioned for the sake of explanation. First off, it is remarkable how it manages to come up with a solution that would be the most obvious to a typical human being, and how when posed with a problem a human being would have difficulty solving, it too had similar difficulty. I find it rather delightful, even if (as he admits) it is not a perfect model of the human thought process. Some of his shortcomings, I feel, better reflect human thought processes though. After all, when stated with the fact that 12x12=9x16, it is apparent, but it is by no means a conclusion that someone would just randomly generate in an attempt to solve one of these problems (or any other problem that doesn't rely wholly on factoring). But that sort of data does represent the statement made, that the intelligence that the program seems to have is based on tiny elements of no real cognitive consequence, merely observations and known facts with straightforward relationships.
Ability to make the jumps to higher ordered variants of lower level observations (such as the 2+2 and 20+20) would certainly make for a more interesting use of the program, especially if it was given only the most bare knowledge of operations to begin with, and then had to build the rest of its pNet from those lower conclusions, and then reused those conclusions on subsequent problems. Even more interesting is if it was only given single digit addition, and had to come to the observations of subtraction, multiplication, division, and perhaps even higher order function from only that basic of conclusion. Perhaps with the brute processing power that computers contain, the program could show a theoretical evolution of mathematics as it recombines its knowledge to create the tools needed to solve a given problem. Certainly, the progression from addition to multiplication to exponents would be reasonable and obvious for it to create, but would it come up with logarithms or derivatives? Or perhaps it would come up with something else entirely, something that would leave us humans thinking "I wish I thought of that".
Ability to make the jumps to higher ordered variants of lower level observations (such as the 2+2 and 20+20) would certainly make for a more interesting use of the program, especially if it was given only the most bare knowledge of operations to begin with, and then had to build the rest of its pNet from those lower conclusions, and then reused those conclusions on subsequent problems. Even more interesting is if it was only given single digit addition, and had to come to the observations of subtraction, multiplication, division, and perhaps even higher order function from only that basic of conclusion. Perhaps with the brute processing power that computers contain, the program could show a theoretical evolution of mathematics as it recombines its knowledge to create the tools needed to solve a given problem. Certainly, the progression from addition to multiplication to exponents would be reasonable and obvious for it to create, but would it come up with logarithms or derivatives? Or perhaps it would come up with something else entirely, something that would leave us humans thinking "I wish I thought of that".
Monday, October 5, 2009
Does this make me a Numbler? (up to p138)
With another section over, the subject matter being covered has changed, or to be more exact, evolved. Jumbo has given way to Numbo, a program that could very well be seen as a variation on a theme as far as purpose goes. Instead of forming english sounding words out of letters, similar techniques are used to solve a problem that is decidedly referred to as a Numble. I've never heard of a Numble, by that name or any other. It is in itself a variant of the crypto problem, the main differences being the much larger range the goal has to the "bricks", the lack of division, and not needing to use all the numbers to come up with a solution.
As far as differences in the implementation of Numbo from Jumbo (other than the obvious difference of numbers and letters), is the concept of a goal. It is here at Numbo that Jumbo and Seek Whence seem to converge, since now there is something to allow top down and bottom up simultaneously, and the other differences introduced in the program's "thought process" reflect this. After all, it makes little sense to just start multiplying randomly the numbers in the set just to see what they get to. I certainly did find myself following many of the thought processes described when looking at the problems presented in the reading, such as looking at two numbers and estimating where the product would fall, roughly.
The most remarkable part of this program would seem to be the way the knowledge base is set up. The excerpt depicted is much how I would envision a semantic network filled with math knowledge would probably look like. Considering the limited domain that this sort of problem fills, and how it was only a small piece of the overall network, it is truly remarkable just how many facts go into trying to solve a given problem, let alone the vast amount of memorized facts that a single human recalls and uses over the course of a waking day. While that is all remarkable in its own right, as the book continually points out, the knowledge is only one side of the coin. A person's ability to draw relevant (as opposed to completely arbitrary) associations between data to solve something is the truly remarkable thing. It is also the thing that varies the most from person to person. A group of students of approximately the same age and (at least in a general sense) the same education will still solve different problems in different ways. When it comes to something in math, perhaps at some point during their education they noticed something that someone else didn't and it made an impression enough that it modified the still forming network in a way that it didn't for someone else, and then later facts building off of it actually causes it to have a completely different end result. Or perhaps that random fact despite it's temporal prominence is just another little branch on the tree after a while.
Of course, the program doesn't actually evolve like this, it is merely programmed. But could it?
As far as differences in the implementation of Numbo from Jumbo (other than the obvious difference of numbers and letters), is the concept of a goal. It is here at Numbo that Jumbo and Seek Whence seem to converge, since now there is something to allow top down and bottom up simultaneously, and the other differences introduced in the program's "thought process" reflect this. After all, it makes little sense to just start multiplying randomly the numbers in the set just to see what they get to. I certainly did find myself following many of the thought processes described when looking at the problems presented in the reading, such as looking at two numbers and estimating where the product would fall, roughly.
The most remarkable part of this program would seem to be the way the knowledge base is set up. The excerpt depicted is much how I would envision a semantic network filled with math knowledge would probably look like. Considering the limited domain that this sort of problem fills, and how it was only a small piece of the overall network, it is truly remarkable just how many facts go into trying to solve a given problem, let alone the vast amount of memorized facts that a single human recalls and uses over the course of a waking day. While that is all remarkable in its own right, as the book continually points out, the knowledge is only one side of the coin. A person's ability to draw relevant (as opposed to completely arbitrary) associations between data to solve something is the truly remarkable thing. It is also the thing that varies the most from person to person. A group of students of approximately the same age and (at least in a general sense) the same education will still solve different problems in different ways. When it comes to something in math, perhaps at some point during their education they noticed something that someone else didn't and it made an impression enough that it modified the still forming network in a way that it didn't for someone else, and then later facts building off of it actually causes it to have a completely different end result. Or perhaps that random fact despite it's temporal prominence is just another little branch on the tree after a while.
Of course, the program doesn't actually evolve like this, it is merely programmed. But could it?
Wednesday, September 30, 2009
On Making up Words (up to p126)
And so we have more Exposition behind the inner workings and reasoning of the Jumbo program. The reason for the title in this case is due to the fact that by the end of the chapter he has introduced an unusual amount of made up terms in the chapter, starting with many utensil based variations of the term spoonerism (something that I have been aware of and very susceptible to for a long time, though I never knew it had a name), and then going to pseudo mathematical concepts like "thinkodynamics". Considering the nature of the program to generate spontaneous english sounding words, this seems more than coincidental. The creation of terminology isn't the issue here though.
I'll be honest, I've never really done jumbles. It's not that I feel I'm not capable of it, it just never really interested me. However, I have played games of scrabble (The competition can be pretty fierce in my house), and the concept of Jumbo almost seems to have a closer application there then it does to jumbles. The main difference is that you know there is an English word in the jumble that uses exactly all the letters. In Scrabble, there probably is one, but it probably doesn't use all the letters, and also has the added complication of having exactly one letter that's already been played on the board. Never mind spacing issues and point values and whatnot.
One sort of thing that isn't included in the program that I tend to do very regularly in scrabble is come up with a combination that is a letter short. Much of it is just wishful thinking, but it's also a very integral part to playing the game, as you can't possible keep track of all the letters that are on the board at a given time, so there is some sort of imaginary blank tile that's always in your set. This seems different, because instead of just dealing with the existing letters, you are doing the same tossing up into the air, but somewhere in the process you have added letters that are not in the set. Toss it up again and a different letter may come down. With those instances where you actually have one or both of the blank tiles, you can be tossing up and coming down with up to 3 completely random letters at any given time. The reason why the methods Hofstadter describes seem most ideal in this situation is the fact that you can't rely purely on your own vocabulary when playing scrabble. There is invariably a dictionary on hand for when what looks like a possible word needs to be double checked. Of course, the word forming ability of pro scrabble players probably forms words that look nothing like english on a fairly regular basis, as letter combinations rarely seen in english (such as words that use a Q without a U after it) are absolutely essential to come out ahead in the game.
Nonetheless, the cognitive model that Jumbo relies on seems like it would apply to scrabble very well. In fact, only little modification would be necessary to make it cover scrabble ideally. But the fact that it is possible to fill in letter combinations that are a few letters short is rather remarkable. The gloms that are so essential to the jumbo system are probably key, but where the extra letters come from is the mystery. Could be completely random, but more likely it's a comparison to the glom rules that the letters come together with, and just picking a letter that seems most ideal. Of course, that letter would still have to be distinguishable from the actual letters, as the breaking down and reforming process that is discussed would still need the option of changing the letter without completely breaking down the word. Of course, I could be completely wrong, but that's how I think it might work.
I'll be honest, I've never really done jumbles. It's not that I feel I'm not capable of it, it just never really interested me. However, I have played games of scrabble (The competition can be pretty fierce in my house), and the concept of Jumbo almost seems to have a closer application there then it does to jumbles. The main difference is that you know there is an English word in the jumble that uses exactly all the letters. In Scrabble, there probably is one, but it probably doesn't use all the letters, and also has the added complication of having exactly one letter that's already been played on the board. Never mind spacing issues and point values and whatnot.
One sort of thing that isn't included in the program that I tend to do very regularly in scrabble is come up with a combination that is a letter short. Much of it is just wishful thinking, but it's also a very integral part to playing the game, as you can't possible keep track of all the letters that are on the board at a given time, so there is some sort of imaginary blank tile that's always in your set. This seems different, because instead of just dealing with the existing letters, you are doing the same tossing up into the air, but somewhere in the process you have added letters that are not in the set. Toss it up again and a different letter may come down. With those instances where you actually have one or both of the blank tiles, you can be tossing up and coming down with up to 3 completely random letters at any given time. The reason why the methods Hofstadter describes seem most ideal in this situation is the fact that you can't rely purely on your own vocabulary when playing scrabble. There is invariably a dictionary on hand for when what looks like a possible word needs to be double checked. Of course, the word forming ability of pro scrabble players probably forms words that look nothing like english on a fairly regular basis, as letter combinations rarely seen in english (such as words that use a Q without a U after it) are absolutely essential to come out ahead in the game.
Nonetheless, the cognitive model that Jumbo relies on seems like it would apply to scrabble very well. In fact, only little modification would be necessary to make it cover scrabble ideally. But the fact that it is possible to fill in letter combinations that are a few letters short is rather remarkable. The gloms that are so essential to the jumbo system are probably key, but where the extra letters come from is the mystery. Could be completely random, but more likely it's a comparison to the glom rules that the letters come together with, and just picking a letter that seems most ideal. Of course, that letter would still have to be distinguishable from the actual letters, as the breaking down and reforming process that is discussed would still need the option of changing the letter without completely breaking down the word. Of course, I could be completely wrong, but that's how I think it might work.
Wednesday, September 23, 2009
More on Glomming (up to p111).
This particular reading was mostly the same topic as the last one, that of his concept of Glomming. The concept seems very much like an extension of the ideas of islands mentioned about Seek Whence. Instead of having numerical runs of different sorts, it's a combination of letters that fit together logically. The Gloms are just combinations of islands like before. The main addition to this system other than the retooling for language is a way of overcoming some of the ambiguity of it, by weighting the different combinations. The most interesting part of the program mentioned so far is the use of randomness. Usually randomness is to be avoided in programming, as its non-systematic nature tends to actually make things longer. And yet, the analogy he uses makes sense, that putting together chemicals in a cell is a mostly random process, but still results in the necessary chemicals. It feels like a fairly effective technique at modeling how we can unscramble words in our minds, with one exception: We do have a dictionary in us, full of thousands of words. While we definitely make combinations in a way similar if not identical to the technique the program relies on, I think we are also constantly matching partial words, and discarding many gloms before the word has fully formed because of that. It works as a synthesis of the patterns found in English, but as a total map of the process used in unscrambling a word it seems to fall short. Of course, he probably has more ideas that will account for that, and it is a very interesting idea whatever the case.
Subscribe to:
Posts (Atom)
