tag:github.com,2008:https://github.com/askui/python-sdk/releasesRelease notes from python-sdk2026-03-18T09:51:49Ztag:github.com,2008:Repository/878401217/v0.27.02026-03-18T09:55:29Zv0.27.0<h1>v0.27.0</h1>
<h2>🎉 Overview</h2>
<p>v0.27.0 adds a MultiDeviceAgent that can operate android and computer devices simultaneously, improves the SDK structure by introducing default tool lists for the ComputerAgent, and AndroidAgent, and fixes a bug with single-display handling on android.</p>
<h2>✨ New Features</h2>
<ul>
<li>Execution Cost in Html Reports by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/philipph-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/philipph-askui">@philipph-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="4046996469" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/243" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/243/hovercard" href="https://github.com/askui/python-sdk/pull/243">#243</a></li>
<li>Otel tracing <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/philipph-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/philipph-askui">@philipph-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="4090334693" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/249" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/249/hovercard" href="https://github.com/askui/python-sdk/pull/249">#249</a></li>
<li>max_steps parameter to stop the execution after a predefined number of steps by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/philipph-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/philipph-askui">@philipph-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="4050720939" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/244" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/244/hovercard" href="https://github.com/askui/python-sdk/pull/244">#244</a></li>
</ul>
<h2>🔧 Improvements</h2>
<ul>
<li>format execution time in html reports as <code>hh:mm:ss</code> by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/philipph-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/philipph-askui">@philipph-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="4069096531" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/247" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/247/hovercard" href="https://github.com/askui/python-sdk/pull/247">#247</a></li>
</ul>
<h2>🐛 Bug Fixes</h2>
<p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/askui/python-sdk/compare/v0.26.1...v0.27.0"><tt>v0.26.1...v0.27.0</tt></a></p>philipph-askuitag:github.com,2008:Repository/878401217/v0.26.12026-03-11T09:45:22Zv0.26.1<h2>What's Changed</h2>
<ul>
<li>calls to the print_to_console_tool are now skipped during cached executions, to save time and tokens by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/philipph-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/philipph-askui">@philipph-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="4052815493" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/245" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/245/hovercard" href="https://github.com/askui/python-sdk/pull/245">#245</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/askui/python-sdk/compare/v0.26.0...v0.26.1"><tt>v0.26.0...v0.26.1</tt></a></p>philipph-askuitag:github.com,2008:Repository/878401217/v0.26.02026-03-10T12:07:47Zv0.26.0<h1>v0.26.0</h1>
<h2>🎉 Overview</h2>
<p>v0.26.0 adds a MultiDeviceAgent that can operate android and computer devices simultaneously, improves the SDK structure by introducing default tool lists for the ComputerAgent, and AndroidAgent, and fixes a bug with single-display handling on android.</p>
<h2>✨ New Features</h2>
<ul>
<li>Add <code>MultiDeviceAgent</code> by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/philipph-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/philipph-askui">@philipph-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="4044735190" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/241" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/241/hovercard" href="https://github.com/askui/python-sdk/pull/241">#241</a></li>
</ul>
<h2>🔧 Improvements</h2>
<ul>
<li>Adds Default tool Lists for Agents by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/philipph-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/philipph-askui">@philipph-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="4046148683" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/242" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/242/hovercard" href="https://github.com/askui/python-sdk/pull/242">#242</a></li>
</ul>
<h2>🐛 Bug Fixes</h2>
<ul>
<li>refactor(android): single-display handling and SingleAndroidDisplay by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/mlikasam-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/mlikasam-askui">@mlikasam-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="4043078618" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/240" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/240/hovercard" href="https://github.com/askui/python-sdk/pull/240">#240</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/askui/python-sdk/compare/v0.25.1...v0.26.0"><tt>v0.25.1...v0.26.0</tt></a></p>philipph-askuitag:github.com,2008:Repository/878401217/v0.25.12026-03-05T15:53:45Zv0.25.1<h2>What's Changed</h2>
<p>the value of <code>model_id</code> for vlm_providers can now be set as env variable by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/philipph-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/philipph-askui">@philipph-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="4029098896" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/239" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/239/hovercard" href="https://github.com/askui/python-sdk/pull/239">#239</a></p>
<p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/askui/python-sdk/compare/v0.25.0...v0.25.1"><tt>v0.25.0...v0.25.1</tt></a></p>philipph-askuitag:github.com,2008:Repository/878401217/v0.25.02026-03-05T13:03:21Zv0.25.0<h1>v0.25.0</h1>
<h2>🎉 Overview</h2>
<p>v0.25.0 is a major release that introduces the new <strong>conversation-based agent architecture</strong> with a pluggable <strong>speaker system</strong>, a complete overhaul of the <strong>caching system</strong>, and a new <strong>callback system</strong> for observing and extending agent behavior.</p>
<p>The agent's control loop has been redesigned around the speaker-conversation pattern, where different "speakers" generate messages while the <code>Conversation</code> class handles tool execution and state management. This enables modular execution strategies — the default <code>AgentSpeaker</code> handles LLM interactions, while the new <code>CacheExecutor</code> speaker replays cached trajectories.</p>
<hr>
<h2>🚨 Breaking Changes</h2>
<h3>Caching Settings Restructured</h3>
<p>The <code>CachingSettings</code> model has been restructured with renamed strategies, nested settings objects, and new defaults.</p>
<p><strong>Strategy renames:</strong></p>
<table>
<thead>
<tr>
<th>Before</th>
<th>After</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>"read"</code></td>
<td><code>"execute"</code></td>
</tr>
<tr>
<td><code>"write"</code></td>
<td><code>"record"</code></td>
</tr>
<tr>
<td><code>"both"</code></td>
<td><code>"auto"</code></td>
</tr>
<tr>
<td><code>"no"</code></td>
<td><code>None</code></td>
</tr>
</tbody>
</table>
<p><strong>Settings structure:</strong></p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="# Before (0.24.x)
from askui.models.shared.settings import CachingSettings
CachingSettings(
strategy="write",
cache_dir=".cache",
filename="login_flow.json",
execute_cached_trajectory_tool_settings=CachedExecutionToolSettings(
delay_time_between_action=0.5,
),
)
# After (0.25.0)
from askui.models.shared.settings import (
CachingSettings,
CacheWritingSettings,
CacheExecutionSettings,
)
CachingSettings(
strategy="record",
cache_dir=".askui_cache", # default changed
writing_settings=CacheWritingSettings(
filename="login_flow.json",
),
execution_settings=CacheExecutionSettings(
delay_time_between_actions=1.0, # default changed from 0.5
),
)"><pre><span class="pl-c"># Before (0.24.x)</span>
<span class="pl-k">from</span> <span class="pl-s1">askui</span>.<span class="pl-s1">models</span>.<span class="pl-s1">shared</span>.<span class="pl-s1">settings</span> <span class="pl-k">import</span> <span class="pl-v">CachingSettings</span>
<span class="pl-en">CachingSettings</span>(
<span class="pl-s1">strategy</span><span class="pl-c1">=</span><span class="pl-s">"write"</span>,
<span class="pl-s1">cache_dir</span><span class="pl-c1">=</span><span class="pl-s">".cache"</span>,
<span class="pl-s1">filename</span><span class="pl-c1">=</span><span class="pl-s">"login_flow.json"</span>,
<span class="pl-s1">execute_cached_trajectory_tool_settings</span><span class="pl-c1">=</span><span class="pl-en">CachedExecutionToolSettings</span>(
<span class="pl-s1">delay_time_between_action</span><span class="pl-c1">=</span><span class="pl-c1">0.5</span>,
),
)
<span class="pl-c"># After (0.25.0)</span>
<span class="pl-k">from</span> <span class="pl-s1">askui</span>.<span class="pl-s1">models</span>.<span class="pl-s1">shared</span>.<span class="pl-s1">settings</span> <span class="pl-k">import</span> (
<span class="pl-v">CachingSettings</span>,
<span class="pl-v">CacheWritingSettings</span>,
<span class="pl-v">CacheExecutionSettings</span>,
)
<span class="pl-en">CachingSettings</span>(
<span class="pl-s1">strategy</span><span class="pl-c1">=</span><span class="pl-s">"record"</span>,
<span class="pl-s1">cache_dir</span><span class="pl-c1">=</span><span class="pl-s">".askui_cache"</span>, <span class="pl-c"># default changed</span>
<span class="pl-s1">writing_settings</span><span class="pl-c1">=</span><span class="pl-en">CacheWritingSettings</span>(
<span class="pl-s1">filename</span><span class="pl-c1">=</span><span class="pl-s">"login_flow.json"</span>,
),
<span class="pl-s1">execution_settings</span><span class="pl-c1">=</span><span class="pl-en">CacheExecutionSettings</span>(
<span class="pl-s1">delay_time_between_actions</span><span class="pl-c1">=</span><span class="pl-c1">1.0</span>, <span class="pl-c"># default changed from 0.5</span>
),
)</pre></div>
<p><strong>Default changes:</strong></p>
<ul>
<li><code>cache_dir</code> default: <code>".cache"</code> → <code>".askui_cache"</code></li>
<li><code>delay_time_between_actions</code> default: <code>0.5</code> → <code>1.0</code> seconds (to give UIs time to materialize)</li>
<li><code>visual_validation_threshold</code> default: changed to <code>10</code></li>
</ul>
<h3><code>act()</code> Parameter Renamed</h3>
<p>The <code>settings</code> parameter on <code>act()</code> has been renamed to <code>act_settings</code>:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="# Before
agent.act("Open settings", settings=ActSettings(...))
# After
agent.act("Open settings", act_settings=ActSettings(...))"><pre><span class="pl-c"># Before</span>
<span class="pl-s1">agent</span>.<span class="pl-c1">act</span>(<span class="pl-s">"Open settings"</span>, <span class="pl-s1">settings</span><span class="pl-c1">=</span><span class="pl-en">ActSettings</span>(...))
<span class="pl-c"># After</span>
<span class="pl-s1">agent</span>.<span class="pl-c1">act</span>(<span class="pl-s">"Open settings"</span>, <span class="pl-s1">act_settings</span><span class="pl-c1">=</span><span class="pl-en">ActSettings</span>(...))</pre></div>
<h3><code>on_message</code> Callback Removed</h3>
<p>The <code>on_message</code> parameter on <code>act()</code> has been removed. Use the new <code>ConversationCallback</code> system instead (see New Features below).</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="# Before
agent.act("Open settings", on_message=my_callback)
# After — use ConversationCallback
from askui import ComputerAgent, ConversationCallback
class LoggingCallback(ConversationCallback):
def on_step_end(self, conversation, step_index, result):
for msg in result.messages_to_add:
print(msg.model_dump_json())
with ComputerAgent(callbacks=[LoggingCallback()]) as agent:
agent.act("Open settings")"><pre><span class="pl-c"># Before</span>
<span class="pl-s1">agent</span>.<span class="pl-c1">act</span>(<span class="pl-s">"Open settings"</span>, <span class="pl-s1">on_message</span><span class="pl-c1">=</span><span class="pl-s1">my_callback</span>)
<span class="pl-c"># After — use ConversationCallback</span>
<span class="pl-k">from</span> <span class="pl-s1">askui</span> <span class="pl-k">import</span> <span class="pl-v">ComputerAgent</span>, <span class="pl-v">ConversationCallback</span>
<span class="pl-k">class</span> <span class="pl-v">LoggingCallback</span>(<span class="pl-v">ConversationCallback</span>):
<span class="pl-k">def</span> <span class="pl-en">on_step_end</span>(<span class="pl-s1">self</span>, <span class="pl-s1">conversation</span>, <span class="pl-s1">step_index</span>, <span class="pl-s1">result</span>):
<span class="pl-k">for</span> <span class="pl-s1">msg</span> <span class="pl-c1">in</span> <span class="pl-s1">result</span>.<span class="pl-c1">messages_to_add</span>:
<span class="pl-en">print</span>(<span class="pl-s1">msg</span>.<span class="pl-c1">model_dump_json</span>())
<span class="pl-k">with</span> <span class="pl-en">ComputerAgent</span>(<span class="pl-s1">callbacks</span><span class="pl-c1">=</span>[<span class="pl-en">LoggingCallback</span>()]) <span class="pl-k">as</span> <span class="pl-s1">agent</span>:
<span class="pl-s1">agent</span>.<span class="pl-c1">act</span>(<span class="pl-s">"Open settings"</span>)</pre></div>
<h3><code>CustomAgent</code> Removed</h3>
<p>The internal <code>CustomAgent</code> class has been removed as part of the architecture refactoring.</p>
<h3>Default Model Updated to <code>claude-sonnet-4-6</code></h3>
<p>The default model for both <code>AnthropicVlmProvider</code> and <code>AskUIVlmProvider</code> has been changed to <code>claude-sonnet-4-6</code> (previously <code>claude-sonnet-4-5-20251101</code> / <code>claude-sonnet-4-5-20250929</code>).</p>
<h3><code>MessageSettings</code> Type Changes</h3>
<p><code>MessageSettings</code> fields now use provider-agnostic types instead of Anthropic-specific ones:</p>
<ul>
<li><code>betas</code> field removed — use <code>provider_options={"betas": [...]}</code> instead</li>
<li><code>thinking</code>, <code>tool_choice</code>: now <code>ThinkingConfigParam | None</code> / <code>ToolChoiceParam | None</code> (instead of Anthropic <code>Beta*</code> types with <code>Omit</code>)</li>
<li><code>temperature</code>: now <code>float | None</code> (instead of <code>float | Omit</code>)</li>
<li>New <code>provider_options: dict[str, Any] | None</code> field for provider-specific options</li>
</ul>
<hr>
<h2>✨ New Features</h2>
<h3>Conversation-Based Architecture</h3>
<p>The agent's control loop has been redesigned around the <strong>speaker-conversation pattern</strong>. The <code>Conversation</code> class orchestrates the flow, managing speaker switching, tool execution, message history, and truncation strategies. Speakers are pluggable message generators that produce responses without executing tools.</p>
<h3>Speaker Handoff System</h3>
<p>A new <strong>speaker handoff pattern</strong> enables dynamic switching between execution strategies. Speakers describe their capabilities in the system prompt, and the LLM can initiate handoffs via a <code>switch_speaker</code> tool. This replaces the hardcoded tool-result checks from previous versions.</p>
<ul>
<li><code>Speaker</code> abstract base class with <code>can_handle()</code>, <code>handle_step()</code>, and <code>on_activate()</code> hooks</li>
<li><code>Speakers</code> registry for managing and looking up speakers</li>
<li><code>SwitchSpeakerTool</code> automatically added when non-default speakers are registered</li>
<li>Speaker descriptions injected into system prompt for LLM-initiated handoffs</li>
</ul>
<h3>Callback System</h3>
<p>A new <strong>PyTorch Lightning-style callback system</strong> provides hooks into the conversation lifecycle:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from askui import ComputerAgent, ConversationCallback
class TimingCallback(ConversationCallback):
def on_conversation_start(self, conversation):
self.start = time.time()
def on_step_end(self, conversation, step_index, result):
print(f"Step {step_index}: {result.status}")
def on_conversation_end(self, conversation):
print(f"Total: {time.time() - self.start:.2f}s")
with ComputerAgent(callbacks=[TimingCallback()]) as agent:
agent.act("Open settings")"><pre><span class="pl-k">from</span> <span class="pl-s1">askui</span> <span class="pl-k">import</span> <span class="pl-v">ComputerAgent</span>, <span class="pl-v">ConversationCallback</span>
<span class="pl-k">class</span> <span class="pl-v">TimingCallback</span>(<span class="pl-v">ConversationCallback</span>):
<span class="pl-k">def</span> <span class="pl-en">on_conversation_start</span>(<span class="pl-s1">self</span>, <span class="pl-s1">conversation</span>):
<span class="pl-s1">self</span>.<span class="pl-c1">start</span> <span class="pl-c1">=</span> <span class="pl-s1">time</span>.<span class="pl-c1">time</span>()
<span class="pl-k">def</span> <span class="pl-en">on_step_end</span>(<span class="pl-s1">self</span>, <span class="pl-s1">conversation</span>, <span class="pl-s1">step_index</span>, <span class="pl-s1">result</span>):
<span class="pl-en">print</span>(<span class="pl-s">f"Step <span class="pl-s1"><span class="pl-kos">{</span><span class="pl-s1">step_index</span><span class="pl-kos">}</span></span>: <span class="pl-s1"><span class="pl-kos">{</span><span class="pl-s1">result</span>.<span class="pl-c1">status</span><span class="pl-kos">}</span></span>"</span>)
<span class="pl-k">def</span> <span class="pl-en">on_conversation_end</span>(<span class="pl-s1">self</span>, <span class="pl-s1">conversation</span>):
<span class="pl-en">print</span>(<span class="pl-s">f"Total: <span class="pl-s1"><span class="pl-kos">{</span><span class="pl-s1">time</span>.<span class="pl-c1">time</span>() <span class="pl-c1">-</span> <span class="pl-s1">self</span>.<span class="pl-c1">start</span>:.2f<span class="pl-kos">}</span></span>s"</span>)
<span class="pl-k">with</span> <span class="pl-en">ComputerAgent</span>(<span class="pl-s1">callbacks</span><span class="pl-c1">=</span>[<span class="pl-en">TimingCallback</span>()]) <span class="pl-k">as</span> <span class="pl-s1">agent</span>:
<span class="pl-s1">agent</span>.<span class="pl-c1">act</span>(<span class="pl-s">"Open settings"</span>)</pre></div>
<p>Available hooks: <code>on_conversation_start</code>, <code>on_conversation_end</code>, <code>on_control_loop_start</code>, <code>on_control_loop_end</code>, <code>on_step_start</code>, <code>on_step_end</code>, <code>on_speaker_switch</code>, <code>on_tool_execution_start</code>, <code>on_tool_execution_end</code>.</p>
<h3>Caching V2: CacheManager and CacheExecutor</h3>
<p>The caching system has been rebuilt from the ground up:</p>
<ul>
<li><strong><code>CacheManager</code></strong> replaces the old <code>CacheWriter</code> with support for cache metadata, validation, parameter handling, and recording</li>
<li><strong><code>CacheExecutor</code> speaker</strong> replays cached trajectories as a dedicated speaker, enabling the agent to verify results after playback and handle non-cacheable tools gracefully</li>
<li><strong>Visual validation</strong> during cache replay using perceptual hashing (pHash/aHash) to detect UI state changes</li>
<li><strong>Cache parameters</strong> — dynamic values in trajectories can be parameterized and substituted at replay time</li>
<li><strong>Cache metadata</strong> — cache files now include metadata (validity, token usage, visual validation config)</li>
<li><strong>Usage statistics</strong> for cached executions shown in HTML reports, including original token consumption from when the cache was recorded</li>
</ul>
<h3>CacheWritingSettings and CacheExecutionSettings</h3>
<p>New dedicated settings classes for fine-grained control over cache recording and playback:</p>
<ul>
<li><code>CacheWritingSettings</code>: <code>filename</code>, <code>parameter_identification_strategy</code>, <code>visual_verification_method</code>, <code>visual_validation_region_size</code></li>
<li><code>CacheExecutionSettings</code>: <code>delay_time_between_actions</code>, <code>skip_visual_validation</code>, <code>visual_validation_threshold</code></li>
</ul>
<h3>Conversation ID</h3>
<p>Each conversation execution is now assigned a unique <code>conversation_id</code> (UUID), accessible via <code>conversation.conversation_id</code>.</p>
<hr>
<h2>🔧 Improvements</h2>
<h3>Speaker Architecture Refinements</h3>
<ul>
<li>Speaker <code>name</code> and <code>description</code> are now validated member variables (non-empty enforced at init)</li>
<li><code>AgentSpeaker</code> now returns <code>"continue"</code> status when tool calls are present (semantically accurate)</li>
<li>Speaker switch logic extracted into dedicated <code>_switch_speaker_if_needed()</code> method</li>
<li><code>_handle_result_status</code> renamed to <code>_handle_continue_conversation</code> for clarity</li>
</ul>
<h3>Anthropic Messages API Refactoring</h3>
<ul>
<li><code>_sanitize_message_for_api</code> refactored into <code>from_message_param</code> and <code>from_content_block</code> functions</li>
<li><code>ToolUseBlockParam.visual_representation</code> (internal cache field) is now properly excluded when sending to the Anthropic API</li>
</ul>
<h3>Code Quality</h3>
<ul>
<li>Removed unnecessary try/except blocks in <code>AgentSpeaker</code> and <code>CacheExecutor</code></li>
<li>Added <code>isEnabledFor(logging.DEBUG)</code> guard for costly <code>model_dump()</code> log calls</li>
<li>Tracing span names made consistent with function names</li>
<li>Usage tracking refactored to integrate via callback pattern</li>
<li>Default <code>.askui_cache</code> directory added to <code>.gitignore</code></li>
</ul>
<h3>HTML Reporter</h3>
<ul>
<li>Truncation of long content (base64 image strings) to prevent report flooding</li>
<li>Cache execution statistics section with original token usage hints</li>
<li>Fixed crash for non-cached executions</li>
</ul>
<hr>
<h2>🐛 Bug Fixes</h2>
<ul>
<li>Fixed bug where cached executions were reported as success when they failed</li>
<li>Fixed bug in visual validation during cached execution</li>
<li>Fixed duplicate clipping of coordinates in mouse move</li>
<li>Fixed agent occasionally providing coordinates as strings instead of integers</li>
<li>Fixed bug where <code>AgentMessages</code> did not show up in reports</li>
<li>Fixed agent response status not accurately reflecting tool-call state</li>
<li>Fixed telemetry error when <code>AgentSettings</code> could not be converted to dict</li>
</ul>
<hr>
<h2>📚 Documentation</h2>
<ul>
<li><strong>Updated <code>docs/06_caching.md</code></strong> with new <code>CachingSettings</code> structure, renamed strategies, and corrected defaults</li>
<li><strong>Updated <code>docs/11_callbacks.md</code></strong> with new callback system documentation</li>
<li><strong>Fixed outdated references</strong> across all docs:
<ul>
<li><code>act()</code> parameter <code>settings</code> → <code>act_settings</code></li>
<li><code>on_message</code> callback removed from MCP example</li>
<li>Typo <code>AntrhopicImageQAProvider</code> → <code>AnthropicImageQAProvider</code></li>
<li><code>vlm_provider=GoogleImageQAProvider</code> → <code>image_qa_provider=GoogleImageQAProvider</code></li>
<li>Broken link to non-existent <code>11_file_support.md</code> fixed</li>
<li><code>_on_speaker_switch</code> → <code>on_speaker_switch</code> in callback hooks table</li>
</ul>
</li>
</ul>
<hr>
<h2>📈 Statistics</h2>
<ul>
<li>73 files changed</li>
<li>5,136 additions</li>
<li>1,500 deletions</li>
</ul>
<h2>Dependencies</h2>
<ul>
<li><code>opentelemetry-sdk</code> made a default dependency (previously optional)</li>
<li>OpenTelemetry instrumentors imported safely with fallback</li>
</ul>
<hr>
<p><strong>Full Changelog</strong>: <a href="https://github.com/askui/python-sdk/compare/v0.24.1...v0.25.0">v0.24.1...v0.25.0</a></p>philipph-askuitag:github.com,2008:Repository/878401217/v0.24.12026-03-03T16:14:45Zv0.24.1<p>fixes a bug where AgentMessages did not show up in reports anymore](<a class="commit-link" data-hovercard-type="commit" data-hovercard-url="https://github.com/askui/python-sdk/commit/75145915670c3c7c78a5ad58c829e78d6175aa1d/hovercard" href="https://github.com/askui/python-sdk/commit/75145915670c3c7c78a5ad58c829e78d6175aa1d"><tt>7514591</tt></a>)</p>
<p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/askui/python-sdk/compare/v0.24.0...v0.24.1"><tt>v0.24.0...v0.24.1</tt></a></p>philipph-askuitag:github.com,2008:Repository/878401217/v0.24.02026-03-03T10:30:37Zv0.24.0<h1>v0.24.0</h1>
<h2>🎉 Overview</h2>
<p>v0.24.0 is a major architectural release that fundamentally refactors how custom models can be integrated into the AskUI SDK. The centerpiece is the new <strong>Bring-Your-Own-Model-Provider</strong> system: instead of configuring models via string identifiers and using a complicated <code>ModelRouter</code> abstraction, you can now simply plug typed provider instances directly into <code>AgentSettings</code>. Three clean interfaces, <code>VlmProvider</code>, <code>ImageQAProvider</code>, and <code>DetectionProvider</code>, make it straightforward to swap in your own model backends for acting, querying, and locating UI elements. Built-in providers for Anthropic and Google are included alongside the AskUI defaults.</p>
<h2>🚨 Breaking Changes</h2>
<ul>
<li>
<p><strong>Model Provider Overhaul:</strong> We removed the <code>ModelRouter</code> and <code>ModelRegistry</code> and replaced them with a new <code>model_providers</code> architecture. You can now bring your own model providers through three typed interfaces: <code>VlmProvider</code> (for <code>act</code>), <code>ImageQAProvider</code> (for <code>get</code>), and <code>DetectionProvider</code> (for <code>locate</code>). Built-in providers include <code>AnthropicVlmProvider</code>, <code>GoogleImageQAProvider</code>, <code>AskUIVlmProvider</code>, and more. Please see the new <a href="https://github.com/askui/python-sdk/blob/main/docs/05_bring_your_own_model_provider.md">Bring Your Own Model Provider docs</a> for detailed instructions.</p>
<p><strong>Migration:</strong> Replace <code>model</code>/<code>models</code> constructor parameters with the new <code>settings: AgentSettings</code> parameter:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="# Before
agent = VisionAgent(model="claude-sonnet-4-20250514")
# After
from askui import ComputerAgent, AgentSettings
from askui.model_providers import AnthropicVlmProvider
agent = ComputerAgent(settings=AgentSettings(
vlm_provider=AnthropicVlmProvider(model_id="claude-sonnet-4-20250514"),
))"><pre><span class="pl-c"># Before</span>
<span class="pl-s1">agent</span> <span class="pl-c1">=</span> <span class="pl-en">VisionAgent</span>(<span class="pl-s1">model</span><span class="pl-c1">=</span><span class="pl-s">"claude-sonnet-4-20250514"</span>)
<span class="pl-c"># After</span>
<span class="pl-k">from</span> <span class="pl-s1">askui</span> <span class="pl-k">import</span> <span class="pl-v">ComputerAgent</span>, <span class="pl-v">AgentSettings</span>
<span class="pl-k">from</span> <span class="pl-s1">askui</span>.<span class="pl-s1">model_providers</span> <span class="pl-k">import</span> <span class="pl-v">AnthropicVlmProvider</span>
<span class="pl-s1">agent</span> <span class="pl-c1">=</span> <span class="pl-en">ComputerAgent</span>(<span class="pl-s1">settings</span><span class="pl-c1">=</span><span class="pl-en">AgentSettings</span>(
<span class="pl-s1">vlm_provider</span><span class="pl-c1">=</span><span class="pl-en">AnthropicVlmProvider</span>(<span class="pl-s1">model_id</span><span class="pl-c1">=</span><span class="pl-s">"claude-sonnet-4-20250514"</span>),
))</pre></div>
</li>
<li>
<p><strong><code>VisionAgent</code> renamed to <code>ComputerAgent</code>:</strong> The main agent class is now <code>ComputerAgent</code>. <code>VisionAgent</code> still works but emits a <code>DeprecationWarning</code>. Similarly, <code>AndroidVisionAgent</code> is now <code>AndroidAgent</code>.</p>
</li>
<li>
<p><strong><code>click()</code>/<code>mouse_move()</code> <code>model</code> parameter replaced:</strong> The <code>model</code> parameter on <code>click()</code>, <code>mouse_move()</code>, and <code>locate()</code> has been replaced by <code>locate_settings: LocateSettings</code> for controlling resolution and other locate options.</p>
</li>
<li>
<p><strong><code>betas</code> parameter removed from <code>MessageSettings</code>:</strong> The Anthropic-specific <code>betas</code> parameter was replaced with a generic <code>provider_options: dict[str, Any]</code> field. To pass betas, use <code>provider_options={"betas": [...]}</code>.</p>
</li>
<li>
<p><strong>Chat API removed:</strong> The Chat API (<code>src/askui/chat/</code>) has been removed from the package along with its dependencies (<code>sqlalchemy</code>, <code>alembic</code>, <code>fastapi</code>, <code>uvicorn</code>, <code>apscheduler</code>, etc.).</p>
</li>
<li>
<p><strong><code>pynput</code> AgentOs backend removed:</strong> The <code>PynputAgentOs</code> implementation and the <code>askui[pynput]</code> optional dependency group have been removed. Use the default <code>AskUiControllerClient</code> (gRPC) backend instead.</p>
</li>
<li>
<p><strong>UITars model removed:</strong> The <code>UITars</code> model integration (<code>src/askui/models/ui_tars_ep/</code>) has been removed.</p>
</li>
<li>
<p><strong>OpenAI integration removed:</strong> The OpenAI-compatible model provider (<code>src/askui/models/openai/</code>) has been removed. Use the new provider interfaces for custom model integrations.</p>
</li>
<li>
<p><strong><code>ModelComposition</code> and <code>ModelDefinition</code> removed:</strong> These classes have been replaced by the new provider system.</p>
</li>
</ul>
<hr>
<h2>✨ New Features</h2>
<ul>
<li>
<p><strong><code>AgentSettings</code> for centralized configuration:</strong> A new <code>AgentSettings</code> class provides a clean, typed configuration surface for agents with three provider slots: <code>vlm_provider</code>, <code>image_qa_provider</code>, and <code>detection_provider</code> — each with sensible AskUI defaults.</p>
</li>
<li>
<p><strong>Bring-Your-Own-Model-Provider:</strong> Three abstract provider interfaces (<code>VlmProvider</code>, <code>ImageQAProvider</code>, <code>DetectionProvider</code>) allow users to plug in their own models. Built-in implementations:</p>
<ul>
<li><code>AskUIVlmProvider</code>, <code>AskUIImageQAProvider</code>, <code>AskUIDetectionProvider</code> (defaults)</li>
<li><code>AnthropicVlmProvider</code>, <code>AnthropicImageQAProvider</code> (direct Anthropic API)</li>
<li><code>GoogleImageQAProvider</code> (direct Google Gemini API)</li>
</ul>
</li>
<li>
<p><strong><code>mouse_movement</code> accepts a <code>duration</code> parameter</strong> to control mouse movement speed (in milliseconds, default: 500ms) by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/philipph-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/philipph-askui">@philipph-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="3957529202" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/233" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/233/hovercard" href="https://github.com/askui/python-sdk/pull/233">#233</a></p>
</li>
<li>
<p><strong>Time and wait tools added to universal tool store</strong> by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/mlikasam-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/mlikasam-askui">@mlikasam-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="3967663860" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/234" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/234/hovercard" href="https://github.com/askui/python-sdk/pull/234">#234</a>:</p>
<ul>
<li><code>GetCurrentTimeTool</code> — returns current date/time for time-aware agent decisions</li>
<li><code>WaitTool</code> — pauses execution for a specified duration</li>
<li><code>WaitWithProgressTool</code> — wait with a visual progress bar</li>
<li><code>WaitUntilConditionTool</code> — polls a condition with configurable interval and timeout</li>
</ul>
</li>
<li>
<p><strong><code>LocateSettings</code> and <code>GetSettings</code> exposed in public API:</strong> Users can now control per-call locate/get behavior including <code>resolution</code>, <code>max_tokens</code>, <code>temperature</code>, and <code>system_prompt</code>.</p>
</li>
<li>
<p><strong><code>FallbackLocateModel</code> and <code>FallbackGetModel</code>:</strong> New utility classes that try multiple models in sequence until one succeeds, replacing the old <code>ModelComposition</code> pattern.</p>
</li>
<li>
<p><strong><code>get</code> and <code>locate</code> tools in act loop:</strong> The LLM can now use <code>get</code> and <code>locate</code> as tools during <code>act()</code> calls (only when an <code>AgentOs</code> is available).</p>
</li>
</ul>
<hr>
<h2>🐛 Bug Fixes</h2>
<ul>
<li>
<p><strong>Fixed agent crash without AgentOs:</strong> <code>get</code> and <code>locate</code> tools are now only added to the act loop when <code>agent_os</code> is set. Agents used without an AgentOs (e.g., pure LLM pipelines) no longer crash on <code>act()</code>. by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/philipph-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/philipph-askui">@philipph-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="4010709193" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/237" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/237/hovercard" href="https://github.com/askui/python-sdk/pull/237">#237</a></p>
</li>
<li>
<p><strong>Fixed OpenTelemetry import errors:</strong> <code>opentelemetry-sdk</code> is now a default dependency. Instrumentor imports (<code>FastAPIInstrumentor</code>, <code>HTTPXClientInstrumentor</code>, etc.) are safely guarded with <code>try/except</code> so installing without <code>[otel]</code> extras no longer causes import failures. by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/philipph-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/philipph-askui">@philipph-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="4011943446" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/238" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/238/hovercard" href="https://github.com/askui/python-sdk/pull/238">#238</a></p>
</li>
<li>
<p><strong>Fixed typechecking issue</strong> in <code>not_given.py</code> — added <code>@final</code> decorator to resolve mypy ambiguity.</p>
</li>
<li>
<p><strong>Fixed <code>Display</code> default value</strong> for <code>name</code> parameter in AgentOS (was raising an error when executing from cache).</p>
</li>
</ul>
<hr>
<h2>📚 Documentation</h2>
<ul>
<li>Complete restructuring of docs (<code>00_overview.md</code> through <code>10_extracting_data.md</code>)</li>
<li>Removed outdated docs for chat API, MCP, and direct tool use</li>
<li>New <a href="https://github.com/askui/python-sdk/blob/main/docs/05_bring_your_own_model_provider.md">Bring Your Own Model Provider</a> guide</li>
<li>Updated reporting docs to distinguish between execution reports and test reports</li>
<li>Updated README to reflect new <code>ComputerAgent</code> class name, corrected Python version requirement (<code>>=3.10, <3.14</code>), and fixed broken links</li>
</ul>
<hr>
<h3>Dependencies</h3>
<p><strong>Removed:</strong> <code>openai</code>, <code>fastapi</code>, <code>uvicorn</code>, <code>sqlalchemy</code>, <code>alembic</code>, <code>apscheduler</code>, <code>pynput</code>, <code>mss</code>, <code>structlog</code>, <code>asgi-correlation-id</code>, <code>starlette-context</code>, <code>anyio</code>, <code>bson</code>, <code>aiofiles</code></p>
<p><strong>Added to core:</strong> <code>opentelemetry-sdk>=1.38.0</code> (promoted from optional <code>chat</code> extras)</p>
<p><strong>Optional extras changed:</strong></p>
<ul>
<li><code>askui[chat]</code> — removed</li>
<li><code>askui[pynput]</code> — removed</li>
<li><code>askui[otel]</code> — now contains only the instrumentor packages (the base SDK is always available)</li>
<li><code>askui[all]</code> — now includes <code>android</code>, <code>bedrock</code>, <code>tracing</code>, <code>vertex</code>, <code>web</code></li>
</ul>
<hr>
<p>📝 <strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/askui/python-sdk/compare/v0.23.1...v0.24.0"><tt>v0.23.1...v0.24.0</tt></a></p>philipph-askuitag:github.com,2008:Repository/878401217/v0.23.12026-02-12T12:32:15Zv0.23.1<h1>Release Notes - v0.23.1</h1>
<h2>🎉 Overview</h2>
<p>This release adds the new <code>LoadImageTool</code>, improves Tool Store tools with flexible paths and encodings, introduces unique instance identifiers for tools, renames the project to AskUI Python SDK, and includes refactoring and documentation fixes. With 44 files changed, 353 additions, and 271 deletions, this is a focused update building on the v0.23.0 foundation.</p>
<h2>✨ New Features</h2>
<h3>LoadImageTool</h3>
<p>New universal tool for loading images from the filesystem:</p>
<ul>
<li><strong>Location</strong>: <code>askui.tools.store.universal.LoadImageTool</code></li>
<li>Loads images from a configurable base directory for analysis, comparison, or visual inspection</li>
<li>Supports common formats (PNG, JPEG etc.)</li>
<li>Use cases: analyzing screenshots, comparing visual elements, verifying image content, providing visual context during execution</li>
</ul>
<p>Use with <code>agent.act()</code> or via <code>act_tools</code> on the agent constructor.</p>
<h3>Unique Tool Instance Identifiers</h3>
<p>Tools now get unique instance identifiers so that multiple instances of the same tool (e.g. different <code>base_dir</code>) can be used together without name collisions. Tool names are derived from base name, optional tags, and a unique ID, and are sanitized and truncated to 64 characters for model compatibility.</p>
<h2>🔧 Improvements</h2>
<h3>Tool Store – File Tools Flexibility</h3>
<ul>
<li><strong>ListFilesTool</strong>, <strong>ReadFromFileTool</strong>, <strong>WriteToFileTool</strong>: <code>base_dir</code> now accepts <code>str | Path</code> (previously <code>str</code> only). Paths are normalized to absolute for consistent descriptions and behavior.</li>
<li><strong>ReadFromFileTool</strong>: New optional <code>encodings</code> parameter (default: <code>["utf-8", "latin-1"]</code>) to try multiple encodings when reading text files. Improves handling of files that are not UTF-8.</li>
<li><strong>WriteToFileTool</strong>: Base directory is no longer created in <code>__init__</code>; creation happens on write, aligning behavior with other file tools.</li>
</ul>
<h3>AskUI Controller Refactoring</h3>
<ul>
<li>Stub checks in <code>AskUiControllerClient</code> are centralized in a <code>_get_stub()</code> helper, removing repeated assert blocks and clarifying error messages (e.g. "Call <code>connect()</code> first").</li>
</ul>
<h3>Documentation and Branding</h3>
<ul>
<li>Project and docs renamed from "AskUI Vision Agent" to <strong>AskUI Python SDK</strong> (README, docs, and HTML report title).</li>
<li>Fixed Custom Models link in README to point to <code>docs/using-models.md#using-custom-models</code> instead of <code>docs/custom-models.md</code>.</li>
<li>Doc and README references updated for consistency.</li>
</ul>
<h2>🐛 Bug Fixes</h2>
<ul>
<li>Fixed stub initialization error message in AskUI controller ("Connect" → "connect()").</li>
<li>Improved file reading robustness via configurable encodings in <code>ReadFromFileTool</code> (avoids <code>UnicodeDecodeError</code> for non-UTF-8 files when an alternative encoding is suitable).</li>
</ul>
<h2>📚 Documentation</h2>
<ul>
<li>README and docs updated for AskUI Python SDK naming and correct Custom Models link.</li>
<li><code>LoadImageTool</code> documented with examples in its docstring and exported from <code>askui.tools.store.universal</code>.</li>
</ul>
<h2>📊 Statistics</h2>
<ul>
<li><strong>44 files changed</strong></li>
<li><strong>353 lines added</strong></li>
<li><strong>271 lines removed</strong></li>
<li><strong>Net change: +82 lines</strong></li>
</ul>
<h2>🚀 New Tools</h2>
<h3>Using LoadImageTool</h3>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from askui import VisionAgent
from askui.tools.store.universal import LoadImageTool
with VisionAgent(act_tools=[LoadImageTool(base_dir="./images")]) as agent:
agent.act("Describe the logo image called './images/logo.png'")"><pre><span class="pl-k">from</span> <span class="pl-s1">askui</span> <span class="pl-k">import</span> <span class="pl-v">VisionAgent</span>
<span class="pl-k">from</span> <span class="pl-s1">askui</span>.<span class="pl-s1">tools</span>.<span class="pl-s1">store</span>.<span class="pl-s1">universal</span> <span class="pl-k">import</span> <span class="pl-v">LoadImageTool</span>
<span class="pl-k">with</span> <span class="pl-en">VisionAgent</span>(<span class="pl-s1">act_tools</span><span class="pl-c1">=</span>[<span class="pl-en">LoadImageTool</span>(<span class="pl-s1">base_dir</span><span class="pl-c1">=</span><span class="pl-s">"./images"</span>)]) <span class="pl-k">as</span> <span class="pl-s1">agent</span>:
<span class="pl-s1">agent</span>.<span class="pl-c1">act</span>(<span class="pl-s">"Describe the logo image called './images/logo.png'"</span>)</pre></div>
<p>Or per-act:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="with VisionAgent() as agent:
agent.act(
"Describe the logo image called './images/logo.png'",
tools=[LoadImageTool(base_dir="./images")]
)"><pre><span class="pl-k">with</span> <span class="pl-en">VisionAgent</span>() <span class="pl-k">as</span> <span class="pl-s1">agent</span>:
<span class="pl-s1">agent</span>.<span class="pl-c1">act</span>(
<span class="pl-s">"Describe the logo image called './images/logo.png'"</span>,
<span class="pl-s1">tools</span><span class="pl-c1">=</span>[<span class="pl-en">LoadImageTool</span>(<span class="pl-s1">base_dir</span><span class="pl-c1">=</span><span class="pl-s">"./images"</span>)]
)</pre></div>
<h2>📝 Full Changelog</h2>
<p>For a complete list of changes, see the <a href="https://github.com/askui/vision-agent/compare/v0.23.0...v0.23.1">git log</a>.</p>
<hr>
<p><strong>Upgrade</strong>: <code>pip install --upgrade askui</code></p>
<p><strong>Documentation</strong>: <a href="https://docs.askui.com" rel="nofollow">docs.askui.com</a></p>mlikasam-askuitag:github.com,2008:Repository/878401217/v0.23.02026-01-22T10:26:29Zv0.23.0<h1>Release Notes - v0.23.0</h1>
<h2>🎉 Overview</h2>
<p>This release introduces a major overhaul of the prompting system, a new Tool Store for extending agent capabilities, automatic AgentOS injection, and numerous improvements and bug fixes. With 106 files changed, 5,972 additions, and 1,767 deletions, this is one of the most significant updates to the Vision Agent.</p>
<h2>✨ New Features</h2>
<h3>Advanced Prompting System</h3>
<p>A completely redesigned prompting paradigm with a structured 5-fold (plus optional 6th) system prompt architecture:</p>
<ul>
<li><strong>System Capabilities</strong>: Defines what the agent can do and how it should behave</li>
<li><strong>Device Information</strong>: Provides platform-specific context (desktop, mobile, web)</li>
<li><strong>UI Information</strong>: Custom information about your specific UI (strongly recommended)</li>
<li><strong>Report Format</strong>: Specifies how to format execution results</li>
<li><strong>Cache Use</strong>: New optional prompt part that specifies when and how the agent should use cache files</li>
<li><strong>Additional Rules</strong>: Optional special handling for edge cases or known issues</li>
</ul>
<p>The new system provides better structure, flexibility, and control over agent behavior. A comprehensive <a href="/askui/python-sdk/blob/v0.23.0/docs/prompting.md">prompting guide</a> has been added to help you create effective custom prompts.</p>
<p><strong>Breaking Change</strong>: System prompts passed as strings will now show a deprecation warning. Use <code>ActSystemPrompt</code> instead.</p>
<h3>Tool Store</h3>
<p>Introducing a new Tool Store that provides optional, extensible tools organized by category:</p>
<ul>
<li>
<p><strong>Universal Tools</strong> (<code>askui.tools.store.universal</code>): Work with any agent type</p>
<ul>
<li><code>ListFilesTool</code>: List files in a directory</li>
<li><code>ReadFromFileTool</code>: Read content from files</li>
<li><code>WriteToFileTool</code>: Write content to files</li>
<li><code>PrintToConsoleTool</code>: Print messages to console during execution</li>
</ul>
</li>
<li>
<p><strong>Computer Tools</strong> (<code>askui.tools.store.computer</code>): Require ComputerAgentOs</p>
<ul>
<li><code>ComputerSaveScreenshotTool</code>: Save screenshots during execution</li>
<li>Experimental window management tools:
<ul>
<li><code>AddWindowAsVirtualDisplayTool</code></li>
<li><code>ListProcessWindowsTool</code></li>
<li><code>ListProcessTool</code></li>
<li><code>SetProcessInFocusTool</code></li>
<li><code>SetWindowInFocusTool</code></li>
</ul>
</li>
</ul>
</li>
<li>
<p><strong>Android Tools</strong> (<code>askui.tools.store.android</code>): Require AndroidAgentOs</p>
<ul>
<li><code>AndroidSaveScreenshotTool</code>: Save screenshots from Android devices</li>
</ul>
</li>
</ul>
<p>Tools can be passed to <code>agent.act()</code> or to the agent constructor as <code>act_tools</code> for persistent availability.</p>
<h3>AgentOS Auto-Injection</h3>
<p>Tools that require AgentOS (like computer or Android tools) now automatically receive the appropriate AgentOS instance. This simplifies tool usage and eliminates the need for manual AgentOS management in most cases.</p>
<h3>Computer Tools Refactoring</h3>
<p>Computer tools have been completely refactored and reorganized:</p>
<ul>
<li>Tools are now properly modularized in <code>askui.tools.computer</code></li>
<li>Removed deprecated <code>AskUiComputerBaseTool</code></li>
<li>Improved separation of concerns and better code organization</li>
<li>New tools added:
<ul>
<li><code>GetSystemInfoTool</code>: Retrieve system information</li>
<li><code>GetActiveProcessTool</code>: Get information about the active process</li>
<li><code>ConnectTool</code> and <code>DisconnectTool</code>: Manage computer connections in chat</li>
</ul>
</li>
</ul>
<h3>Enhanced AgentOS Capabilities</h3>
<ul>
<li>Updated AgentOS JSON schema with expanded capabilities</li>
<li>New system information retrieval methods</li>
<li>Improved window management capabilities</li>
<li>Better error handling for gRPC invalid argument errors</li>
</ul>
<h3>SSL Verification Control</h3>
<p>Added the ability to disable SSL verification for the user identification API via the <code>ASKUI_HTTP_SSL_VERIFICATION</code> environment variable. This is useful for development environments with self-signed certificates.</p>
<p><strong>Note</strong>: SSL verification is enabled by default for security.</p>
<h2>🔧 Improvements</h2>
<h3>Chat API Enhancements</h3>
<ul>
<li>Added computer connect/disconnect tools for chat interface</li>
<li>Improved chat history management</li>
<li>Enhanced MCP server integration for computer operations</li>
</ul>
<h3>Agent Improvements</h3>
<ul>
<li>Cleaned up agent and agent_base code</li>
<li>Fixed typechecking bugs</li>
<li>Improved reporter encoding handling</li>
<li>Added more reporter messages for better observability</li>
<li>Enhanced overlay support during e2e controller tests</li>
</ul>
<h3>Prompting Improvements</h3>
<ul>
<li>Refined how system prompts are provided</li>
<li>Introduced <code>cache_use</code> prompt part for better cache control</li>
<li>Improved prompt structure and organization</li>
<li>Better handling of prompt parts</li>
</ul>
<h3>Tool System Improvements</h3>
<ul>
<li>Better tool organization and categorization</li>
<li>Improved tool initialization and lifecycle management</li>
<li>Enhanced tool tagging system</li>
<li>Better support for tools with AgentOS requirements</li>
</ul>
<h2>🐛 Bug Fixes</h2>
<ul>
<li>Fixed act prompts issues</li>
<li>Fixed reporter encoding problems</li>
<li>Fixed tool initialization bugs</li>
<li>Fixed typechecking issues in agent</li>
<li>Fixed linter issues across the codebase</li>
<li>Fixed typos in documentation and code</li>
</ul>
<h2>📚 Documentation</h2>
<ul>
<li>Added comprehensive <a href="/askui/python-sdk/blob/v0.23.0/docs/prompting.md">System Prompts documentation</a></li>
<li>Updated README with Tool Store examples</li>
<li>Improved code examples and usage patterns</li>
</ul>
<h2>🔄 Code Quality</h2>
<ul>
<li>Extensive code cleanup and refactoring</li>
<li>Improved type hints and type safety</li>
<li>Better code organization and structure</li>
<li>Enhanced test coverage</li>
</ul>
<h2>📊 Statistics</h2>
<ul>
<li><strong>106 files changed</strong></li>
<li><strong>5,972 lines added</strong></li>
<li><strong>1,767 lines removed</strong></li>
<li><strong>Net change: +4,205 lines</strong></li>
</ul>
<h2><g-emoji class="g-emoji" alias="warning">⚠️</g-emoji> Breaking Changes</h2>
<ol>
<li>
<p><strong>System Prompt Format</strong>: System prompts should now use <code>ActSystemPrompt</code> instead of plain strings. Passing strings will show a deprecation warning.</p>
</li>
<li>
<p><strong>Computer Tools</strong>: The <code>AskUiComputerBaseTool</code> has been removed. Use tools from <code>askui.tools.computer</code> or <code>askui.tools.store.computer</code> instead.</p>
</li>
<li>
<p><strong>Tool Organization</strong>: Computer tools have been reorganized. If you were using tools directly from <code>askui.tools.computer</code>, check the new structure.</p>
</li>
</ol>
<h2>🚀 Migration Guide</h2>
<h3>Using the Tool Store</h3>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from askui import VisionAgent
from askui.tools.store.universal import PrintToConsoleTool, WriteToFileTool
from askui.tools.store.computer import ComputerSaveScreenshotTool
with VisionAgent(act_tools=[
PrintToConsoleTool(),
WriteToFileTool(base_dir="./output"),
ComputerSaveScreenshotTool(base_dir="./screenshots")
]) as agent:
agent.act("Take a screenshot and save it")"><pre><span class="pl-k">from</span> <span class="pl-s1">askui</span> <span class="pl-k">import</span> <span class="pl-v">VisionAgent</span>
<span class="pl-k">from</span> <span class="pl-s1">askui</span>.<span class="pl-s1">tools</span>.<span class="pl-s1">store</span>.<span class="pl-s1">universal</span> <span class="pl-k">import</span> <span class="pl-v">PrintToConsoleTool</span>, <span class="pl-v">WriteToFileTool</span>
<span class="pl-k">from</span> <span class="pl-s1">askui</span>.<span class="pl-s1">tools</span>.<span class="pl-s1">store</span>.<span class="pl-s1">computer</span> <span class="pl-k">import</span> <span class="pl-v">ComputerSaveScreenshotTool</span>
<span class="pl-k">with</span> <span class="pl-en">VisionAgent</span>(<span class="pl-s1">act_tools</span><span class="pl-c1">=</span>[
<span class="pl-en">PrintToConsoleTool</span>(),
<span class="pl-en">WriteToFileTool</span>(<span class="pl-s1">base_dir</span><span class="pl-c1">=</span><span class="pl-s">"./output"</span>),
<span class="pl-en">ComputerSaveScreenshotTool</span>(<span class="pl-s1">base_dir</span><span class="pl-c1">=</span><span class="pl-s">"./screenshots"</span>)
]) <span class="pl-k">as</span> <span class="pl-s1">agent</span>:
<span class="pl-s1">agent</span>.<span class="pl-c1">act</span>(<span class="pl-s">"Take a screenshot and save it"</span>)</pre></div>
<h2>📝 Full Changelog</h2>
<p>For a complete list of changes, see the <a href="https://github.com/askui/vision-agent/compare/v0.22.12...v0.23.0">git log</a>.</p>
<hr>
<p><strong>Upgrade</strong>: <code>pip install --upgrade askui</code></p>
<p><strong>Documentation</strong>: <a href="https://docs.askui.com" rel="nofollow">docs.askui.com</a></p>mlikasam-askuitag:github.com,2008:Repository/878401217/v0.22.122026-01-07T10:13:13Zv0.22.12<h2>What's Changed</h2>
<ul>
<li>feat: add SBOM generation and release workflow by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/mlikasam-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/mlikasam-askui">@mlikasam-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="3756949006" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/214" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/214/hovercard" href="https://github.com/askui/python-sdk/pull/214">#214</a></li>
<li>Add SBOM generator by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/mlikasam-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/mlikasam-askui">@mlikasam-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="3760318881" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/215" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/215/hovercard" href="https://github.com/askui/python-sdk/pull/215">#215</a></li>
<li>ci: ensure SBOM generation runs after PyPI publish by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/mlikasam-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/mlikasam-askui">@mlikasam-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="3760535576" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/216" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/216/hovercard" href="https://github.com/askui/python-sdk/pull/216">#216</a></li>
<li>refactor: remove functools.cache decorator from create_api_client fun… by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/danyalxahid-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/danyalxahid-askui">@danyalxahid-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="3770862482" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/218" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/218/hovercard" href="https://github.com/askui/python-sdk/pull/218">#218</a></li>
<li>Cl 1935 scheduling workflows which run on a specific time by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/danyalxahid-askui/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/danyalxahid-askui">@danyalxahid-askui</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="3760730359" data-permission-text="Title is private" data-url="https://github.com/askui/python-sdk/issues/217" data-hovercard-type="pull_request" data-hovercard-url="/askui/python-sdk/pull/217/hovercard" href="https://github.com/askui/python-sdk/pull/217">#217</a></li>
</ul>
<h2>🚀 New Features</h2>
<ul>
<li>Background Scheduler for Executing Run</li>
<li>Create SBOM</li>
</ul>
<h2>🐛 Bug Fixes</h2>
<ul>
<li>Unauthorized issues with long Running instances</li>
</ul>
<p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/askui/python-sdk/compare/v0.22.11...v0.22.12"><tt>v0.22.11...v0.22.12</tt></a></p>danyalxahid-askui