Class MarkovText

java.lang.Object
com.github.yellowstonegames.text.MarkovText
public class MarkovText extends Object
A simple Markov chain text generator; call analyze(CharSequence) once on a large sample text, then you can call chain(long) many times to get odd-sounding "remixes" of the sample text. This is an order-2 Markov chain, so it chooses the next word based on the previous two words. This is meant to allow easy serialization of the necessary data to call chain(); if you can store the words and processed arrays in some serialized form, then you can reassign them to the same fields to avoid calling analyze(). One way to do this conveniently is to use stringSerialize() after calling analyze() once and to save the resulting String; then, rather than calling analyze() again on future runs, you would call stringDeserialize(String) to create the MarkovText without needing any repeated analysis.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    com.github.tommyettinger.ds.IntIntMap
    pairs
    Map of all pairs of words encountered to the position in the order they were encountered.
    int[][]
    processed
    Complicated data that mixes probabilities of words using their indices in words and the indices of word pairs in pairs, generated during the latest call to analyze(CharSequence).
    String[]
    words
    All words (case-sensitive and counting some punctuation as part of words) that this encountered during the latest call to analyze(CharSequence).

  • Constructor Summary

    Constructors
    Constructor
    Description
    MarkovText()
    Creates an empty MarkovText; you should call analyze(CharSequence) before doing anything else with this new object.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    analyze(CharSequence corpus)
    This is the main necessary step before using a MarkovText; you must call this method at some point before you can call any other methods.
    String
    chain(long seed)
    Generate a roughly-sentence-sized piece of text based on the previously analyzed corpus text (using analyze(CharSequence)) that terminates when stop punctuation is used (".", "!", "?", or "..."), or once the length would be greater than 200 characters without encountering stop punctuation(it terminates such a sentence with "." or "...").
    String
    chain(long seed, int maxLength)
    Generate a roughly-sentence-sized piece of text based on the previously analyzed corpus text (using analyze(CharSequence)) that terminates when stop punctuation is used (".", "!", "?", or "...") or once the maxLength would be exceeded by any other words (it terminates such a sentence with "." or "...").
    void
    changeNames(Translator translator)
    After calling analyze(CharSequence), you can optionally call this to alter any words in this MarkovText that were used as a proper noun (determined by whether they were capitalized in the middle of a sentence), changing them to a ciphered version using the given Translator.
    void
    changeNames(Translator translator, Collection<? extends CharSequence> names)
    After calling analyze(CharSequence), you can optionally call this to alter any words in this MarkovText that are present in the given Collection, changing them to a ciphered version using the given Translator.
    MarkovText
    copy()
    Copies the String array words and the 2D jagged int array processed into a new MarkovText.
    static MarkovText
    stringDeserialize(String data)
    Recreates an already-analyzed MarkovText given a String produced by stringSerialize().
    String
    stringSerialize()
    Returns a representation of this MarkovText as a String; use stringDeserialize(String) to get a MarkovText back from this String.

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • words

      public String[] words
      All words (case-sensitive and counting some punctuation as part of words) that this encountered during the latest call to analyze(CharSequence). Will be null if analyze(CharSequence) was never called.

    • pairs

      public com.github.tommyettinger.ds.IntIntMap pairs
      Map of all pairs of words encountered to the position in the order they were encountered. Pairs are stored using their 16-bit words indices placed into the most-significant bits for the first word and the least-significant bits for the second word. The size of this IntIntMap is likely to be larger than the String array words, but should be equal to processed.length. Will be null if analyze(CharSequence) was never called.

    • processed

      public int[][] processed
      Complicated data that mixes probabilities of words using their indices in words and the indices of word pairs in pairs, generated during the latest call to analyze(CharSequence). This is a jagged 2D array. Will be null if analyze(CharSequence) was never called.

  • Constructor Details

    • MarkovText

      public MarkovText()
      Creates an empty MarkovText; you should call analyze(CharSequence) before doing anything else with this new object.

  • Method Details

    • analyze

      public void analyze(CharSequence corpus)
      This is the main necessary step before using a MarkovText; you must call this method at some point before you can call any other methods. You can serialize this MarkovText after calling to avoid needing to call this again on later runs, or even include serialized MarkovText objects with a game to only need to call this during pre-processing. This method analyzes the pairings of words in a (typically large) corpus text, including some punctuation as part of words and some kinds as their own "words." It only uses one preceding word to determine the subsequent word. When it finishes processing, it stores the results in words and processed, which allows other methods to be called (they will throw a NullPointerException if analyze() hasn't been called).
      Parameters:
      corpus - a typically-large sample text in the style that should be mimicked

    • changeNames

      public void changeNames(Translator translator)
      After calling analyze(CharSequence), you can optionally call this to alter any words in this MarkovText that were used as a proper noun (determined by whether they were capitalized in the middle of a sentence), changing them to a ciphered version using the given Translator. Normally you would initialize a Translator with a Language that matches the style you want for all names in this text, then pass that to this method during pre-processing (not necessarily at runtime, since this method isn't especially fast if the corpus was large). This method modifies this MarkovText in-place.
      Parameters:
      translator - a Translator that will be used to translate proper nouns in this MarkovText's word array

    • changeNames

      public void changeNames(Translator translator, Collection<? extends CharSequence> names)
      After calling analyze(CharSequence), you can optionally call this to alter any words in this MarkovText that are present in the given Collection, changing them to a ciphered version using the given Translator. Normally you would initialize a Translator with a Language that matches the style you want for all names in this text, then pass that to this method during pre-processing (not necessarily at runtime, since this method isn't especially fast if the corpus was large). This method modifies this MarkovText in-place.
      A good way to use this when you have some group of names, but might encounter them in different cases (such as ALL CAPS in a header, or Capitalized Like A Name), is to use a CaseInsensitiveSet as the Collection. It won't handle punctuation next to a word, though; a somewhat different approach is needed for that.
      Parameters:
      translator - a Translator that will be used to translate proper nouns in this MarkovText's word array
      names - a Collection of names that, if a word from the word array is also in names, will be translated

    • chain

      public String chain(long seed)
      Generate a roughly-sentence-sized piece of text based on the previously analyzed corpus text (using analyze(CharSequence)) that terminates when stop punctuation is used (".", "!", "?", or "..."), or once the length would be greater than 200 characters without encountering stop punctuation(it terminates such a sentence with "." or "...").
      Parameters:
      seed - the seed for the random decisions this makes, as a long; any long can be used
      Returns:
      a String generated from the analyzed corpus text's word placement, usually a small sentence

    • chain

      public String chain(long seed, int maxLength)
      Generate a roughly-sentence-sized piece of text based on the previously analyzed corpus text (using analyze(CharSequence)) that terminates when stop punctuation is used (".", "!", "?", or "...") or once the maxLength would be exceeded by any other words (it terminates such a sentence with "." or "...").
      Parameters:
      seed - the seed for the random decisions this makes, as a long; any long can be used
      maxLength - the maximum length for the generated String, in number of characters
      Returns:
      a String generated from the analyzed corpus text's word placement, usually a small sentence

    • stringSerialize

      public String stringSerialize()
      Returns a representation of this MarkovText as a String; use stringDeserialize(String) to get a MarkovText back from this String. The words and processed fields must have been given values by either direct assignment, calling analyze(CharSequence), or building this MarkovTest with the aforementioned destringSerialize method. Uses spaces to separate words and a tab to separate the two fields.
      Returns:
      a String that can be used to store the analyzed words and frequencies in this MarkovText

    • stringDeserialize

      public static MarkovText stringDeserialize(String data)
      Recreates an already-analyzed MarkovText given a String produced by stringSerialize().
      Parameters:
      data - a String returned by stringSerialize()
      Returns:
      a MarkovText that is ready to generate text with chain(long)

    • copy

      public MarkovText copy()
      Copies the String array words and the 2D jagged int array processed into a new MarkovText. None of the arrays will be equivalent references, but the Strings (being immutable) will be the same objects in both MarkovText instances. This is primarily useful with changeNames(Translator), which can produce several variants on names given several initial copies produced with this method.
      Returns:
      a copy of this MarkovText