Hack 44 Store a List of All Input Words

Accumulate a list of popular words for the purposes of searching, indexing, or text autocomplete and store it in a local file.

We saw earlier how to autocomplete text [Hack #43] from a preinitialized word list (i.e., a limited dictionary). That works fine for common words, but in the real world, the definition of "common" can vary by situation. For example, the 300 or 3,000 most common words when talking about computers are not the same words used in common conversation (even if all your friends are geeks, they weren't the sample population used to determine the words in the preceding dictionary).

And the most common use for autocompleting text fields is when filling out online forms. These tend to be populated with uncommon words specific to the end user, such as her name and the name of her street and city.

So this hack shows how to dynamically build a list of words based on those that have been entered previously (which by definition means they're popular in your neck of the woods).

Add New Words to the Dictionary

We want to consider looking for new words only if the current text input has been completed by the user. On a form, this would occur when the user clicks a Submit button to indicate she has finished.

As part of the onRelease( ) event handler of this Submit button, we can search the text entered in our text field(s) and add to our dictionary any words not already in it.

The Code

The following code achieves this (assuming the code in [Hack #43] is also present and that the Submit button's onRelease( ) event handler invokes the function entered( )).

function entered( ) {

  var newWords:Array = new Array( );

  newWords = myText_txt.text.split(" ");

  for (var i = 0; i < newWords.length; i++) {

    if (myText.indexOf(newWords[i].toString( )) == -1) {

      // New word not in dictionary,

      // so add it and re-sort the dictionary.

      myText += " " + newWords[i];

      dictionary.push(newWords[i]);

      dictionary.sort( );

  }

}

Lines 2 and 3 of this function create a new array, newWords, consisting of all the words in the text field myText_text. If you had multiple text fields in your form, you could simply concatenate them all into one string with something like this instead of the current line 3:

newWords = (myTextField1_txt.text + " " +

            myTextField2_txt.text).split(" ");

The for loop's body (lines 4-11) searches the dictionary for every word in the text field (you could, of course, limit it to words with three or more letters). If the new word is not found, it is added to the dictionary, and the dictionary is re-sorted.

Although this code uses the text in the string myText for a fast search [Hack #79], it uses the dictionary array to store new words [Hack #43] . You can see this in action if you test the movie in Debug Movie mode (ControlDebug Movie) by looking at the array _root.dictionary as you enter "aardvark." When you click the Submit button, you will see that dictionary[0] now stores the word "aardvark." If you backspace until you have deleted "aardvark" and then start to retype it, Flash autocompletes the word once you enter "aa".

Save the Dictionary for Later Use

Storing the words in Flash variables is great, but the new words will be lost every time the user exits the form. We need to store the dictionary between sessions locally on the user's hard drive. The easiest way to do this is to save the variable myText, which is a string that holds the entire dictionary (including both default and new words).

The following listing shows the code needed to search for new words and store them locally (additions from earlier versions are shown in bold):

function entered( ) {

  var newWords:Array = new Array( );

  var needUpdate:Boolean = false;

  newWords = myText_txt.text.split(" ");

  for (var i = 0; i < newWords.length; i++) {

    if (myText.indexOf(newWords[i].toString( )) == -1) {

      // New word not in dictionary, 

      // so add it and re-sort the dictionary.

      myText += " " + newWords[i];

      dictionary.push(newWords[i]);

      dictionary.sort( );

      needUpdate = true;

    }

  }

  if (needUpdate) {

    saveDictionary( );

  }

}

function saveDictionary( ) {

  var shared:SharedObject = SharedObject.getLocal("dictionary");

  shared.data.dictionary = myText;

  shared.flush( );

}

function loadDictionary( ) {

  var shared:SharedObject = SharedObject.getLocal("dictionary");

  if (shared.data.dictionary != undefined) {

    myText = shared.data.dictionary;

  }

}

function enterEvent( ) {

  entered( );

  // Do other Submit button stuff...

}

function autoComplete( ) {

  if (Key.isDown(Key.CONTROL)) {

    myText_txt.text = complete_txt.text+" ";

    Selection.setSelection(myText_txt.text.length, 

                           myText_txt.text.length);

  }

}

function fieldChange( ) {

  match = "";

  startOfWord = this.text.lastIndexOf(" ") + 1;

  lastWord = this.text.substring(startOfWord, this.text.length);

  if (lastWord.length>1) {

    for (var i = 0; i < dictionary.length; i++) {

      if (lastWord == (dictionary[i].substr(0, lastWord.length))) {

        // match found

        match = dictionary[i];

        search = i;

        break;

      }

    }

  } else {

    search = 0;

  }

  complete_txt.text = this.text.substr(0, startOfWord) + match;

}

// Initialize

// Full list of words not shown

var myText:String = "about above... you young your";

var dictionary:Array = new Array( );

var search:Number = 0;

var lastWord:String = "";

var startOfWord:String = "";

var control:Object = new Object( );

// Load stored dictionary

loadDictionary( );

// Set up dictionary

dictionary = myText.split(" ");

dictionary.sort( );

// Set up events and listeners

myText_txt.onChanged = fieldChange;

control.onKeyDown = autoComplete;

Key.addListener(control);

enter_btn.onRelease = enterEvent;

The third line of the revised entered( ) function defines a Boolean, needUpdate, that is set to true when new words are found that need to be added to the dictionary. The last line of entered( ) calls saveDictionary( ) if new words have been added to the dictionary. The saveDictionary( ) function saves the text contained in the current full dictionary as a local shared object (LSO). An LSO is just a convenient way to store data between sessions akin to a browser cookie. We'll load the data back in when we need it.

When you attempt to store an LSO that exceeds the allowable size on a user's machine, the Flash Player requests the user's permission to store more data by displaying the Local Storage tab of the Settings dialog box. By default, the local storage limit is 100 KB per domain.

If you intend to store data locally, advise the user of its purpose and intended use. If he sees the dialog box appear unexpectedly, he may get very jittery, especially if it appears when filling out a form to purchase something online!

Loading the saved dictionary is accomplished by loadDictionary( ). This looks for a previously saved dictionary in the LSO and, if it exists, the new text is used to replace the default dictionary, myText.

You can test that new words are now persistent between separate sessions by testing the movie and entering "aardvark" in the text field, then closing the SWF file and running it again. The second time, you will see that "aardvark" autocompletes after the second letter.

Final Thoughts

Notice that the final listing searches the dictionary in two ways:

• When the user is typing words in the text field, we search alphabetically through the dictionary array for the purposes of autocompletion.

• When we want to see if any of the newly entered words are in the dictionary, we perform a search using the myText string.

Thus, we have two versions of the list of words, one that allows a structured alphabetical search and one that allows us to say if a newly input word is in our current dictionary. Both searches are optimized for speed.

Also, you may have noticed that we've assumed all words in the word list have equal popularity, or rather that their popularity matches their alphabetical order. That isn't necessarily true. For example, the word "said" is more popular than "sad" but if we arrange our dictionary alphabetically, "sad" will be suggested before "said." One solution is to list terms in order of popularity (either by using the popularity count provided by large statistical analyses or by simply counting their usage in relation to the current application). But this would slow down searching if the list isn't also (at least partially) alphabetical. If the word lists are small, searching should always be relatively fast. If the word lists are large, then the appropriate solution depends on the degree to which the alphabetical word order is counterproductive.

This technique of capturing popular words can be used for more than autocompleting text. It can be used to accumulate statistics to, say, track the most popular answer to a survey question or to suggest popular search terms when no matches can be found for the requested search term. Of course, those more sophisticated uses apply primarily when you are capturing data on the back end server. This hack captures data locally, assuming that each user's commonly entered words are specific to her (such as her name and street address). You could use Flash Remoting or other approaches to upload the data to a server and accumulate statistics across multiple users.