<doc>
  <legacy_id>2182</legacy_id>
  <name>token</name>
  <type>keyword</type>
  <syntax>
  </syntax>
  <library></library>
  <objects>
  </objects>
  <synonyms>
  </synonyms>
  <classification>
    <category>Writing LiveCode</category>
    <category>Text and Data Processing</category>
  </classification>
  <references>
    <keyword tag="word">word Keyword</keyword>
    <keyword tag="real8">real8 Keyword</keyword>
  </references>
  <history>
    <introduced version="1.0">Added.</introduced>
  </history>
  <platforms>
    <mac/>
    <windows/>
    <linux/>
    <ios/>
    <android/>
  </platforms>
  <classes>
    <desktop/>
    <server/>
    <web/>
    <mobile/>
  </classes>
  <security>
  </security>
  <summary>Designates a <glossary tag="LiveCode">LiveCode</glossary> token as part of a <glossary tag="chunk expression">chunk expression</glossary>.</summary>
  <examples>
    <example>get token 1 of "someFunction(a+b)" <i>-- yields someFunction</i></example>
    <example>get token 2 of "someFunction(a+b)" <i>-- yields (</i></example>
    <example>get token 3 of "someFunction(a+b)" <i>-- yields a</i></example>
    <example>get token 3 of "a+b" <i>-- yields b</i></example>
  </examples>
  <description>
    <p>Use the <b>token</b> keyword to parse a LiveCode statement.</p><p/><p><b>Comments:</b></p><p>A token is a string of characters delimited by certain punctuation marks. The rules for deciding what a token is are as follows:</p><p/><p>1. Each of the following characters is a token: =, +, -, *, /, [, ], (, ), {, }, &lt;, &gt;, and comma (,).</p><p/><p>2. Each of the following characters is a token delimiter: ; (semicolon), space, return, and tab.</p><p/><p>3. Each string of any other characters that is either separated by one of the above characters, or delimited by double quotes, is a token. The token does not include the separating character(s).</p><p/><p>A single token can contain multiple characters, but not multiple words, items, or lines.</p><p/><p><b>Note:</b> The token chunk is implemented for the LiveCode language, and probably isn't suitable for use in a general-purpose language parser.</p>
  </description>
</doc>