Gestures · Voice · On device

Draw a sign. The phone obeys.

Sigil pilots your phone with signs drawn in the air and with your voice. Everything it sees and hears stays on the phone.

Download for Android See how it works

Android preview with live hands, voice and an on device model.

  • 21 points per hand
  • Models on the phone
  • No account, no cloud

00The sign

Four ways to read the mark

  1. 01 · A letter

    The S of SIGIL.

    Two strokes around a point of light. The spine is never drawn: the light is the spine.

  2. 02 · A gesture

    Two strokes, one point.

    A fingertip closing on a point, the way a thumb and an index finger meet in a pinch.

  3. 03 · A cursor

    The light is the cursor.

    The point follows your index finger across the screen, and it is the core of the Control Sphere.

  4. 04 · A seal

    Turn it. It stays the same.

    Half a turn leaves the sign unchanged, as a seal should. Its Latin root, sigillum, means a small seal.

01How it works

From a sign in the air to a tap on the screen.

Three stages, and every one of them runs on the phone.

A music app on a phone. The holographic cursor sweeps in as a trail of light and snaps onto the Play button, which lights up as a glass lens.

  1. 01 · See

    The camera sees your hand.

    The front camera finds 21 points on your hand, 30 to 60 times a second. Frames are read in memory and dropped.

  2. 02 · Understand

    Gestures become intents.

    A small classifier reads poses and motion: a pinch, a flick, a circle in the air. The app in front decides what each sign means.

  3. 03 · Act

    Actions run.

    The cursor snaps onto targets, so a pinch lands where you aimed. Anything that sends, posts or pays waits for your yes.

From camera to action

  1. Front camera
  2. 21 hand points
  3. One Euro filter
  4. Gesture classifier
  5. App profile
  6. Action

02The gesture set

14 signs, one hand.

Every sign is drawn palm toward the camera, the way it sees you. Each one also has a touch path and a voice path.

Gesture 03 / 14

Pinch

Recognized when
Thumb and index tips meet, then open again within 300 ms.
What it does
Taps where the cursor is. The press lands where you aimed 90 ms earlier, before your fingers began to close.

Rewind 90 ms

No sign fires while your hand reaches for the screen, while two hands overlap or within 300 ms of the last one. Losing your hand never commits an action.

03The Control Sphere

Open your palm. The sphere answers.

A glass orb opens where your hand is, over the app you are using, with the actions that make sense right now.

The Control Sphere: a glass orb with the point of light at its core, a status ring and six quick actions on an orbit. Done is lit under the cursor, the app sits above and recent people below.

  1. 01 · Summon

    Hold an open palm.

    After 400 ms of stillness the orb blooms where your hand is. Saying “Sigil” works too.

  2. 02 · Offer

    Six actions, picked by the app.

    In Messages: Reply, Read aloud and Done. Recent people sit below the orbit, initials only.

  3. 03 · Choose

    Point. Pinch. Done.

    The cursor snaps onto the action. Experts flick toward it without looking.

Summon
Hold an open palm still for 400 ms, say “Sigil”, or long press the edge of the screen.
Choose
Point and pinch, or pinch the core and flick toward an action without looking. You can also say its name or touch it.
Dismiss
Close your fist, swipe down with two fingers or say “Sigil, close”. With no hand and no voice for 8 seconds, it closes on its own.

The mark in motion

Breathing: one slow breath every 5.2 seconds.

The arcs open by 8 degrees and pulse with your voice. Follow ups need no wake word for 8 seconds.

The arcs orbit the point, one turn every 1.6 seconds. After 1.2 seconds the caption reads “Working on it.”

The point brightens once and settles in 320 ms.

With reduced motion the mark holds still, and the label carries the state.

04Per app profiles

Each app speaks its own language.

Sigil knows which app is in front and loads its profile. A flick moves to the next Reel in Instagram, and a two finger swipe skips to the next video in YouTube. Apps without a profile use the general one.

Three glass cards for Reader, Music and Camera, each mapping gestures to actions for that app. The Music profile is active and lit in front.
FIG. 04The profile switches with the app in front.

Android packagecom.whatsapp, com.whatsapp.w4b

WhatsApp: gestures and their actions
GestureAction
Flick rightSwipe right from the cursor, to reply to that message
Two finger swipe leftSwipe the screen left
Two finger swipe rightSwipe the screen right

Everything else comes from the general profile. Voice plans use the search field, the message box and the send button of WhatsApp, and still stop before sending.

src/app-profiles/profiles/youtube.json
{
  "id": "youtube",
  "packages": ["com.google.android.youtube"],
  "bindings": {
    "flick_up": { "type": "swipe", "direction": "up" },
    "swipe_two_left": {
      "type": "media", "action": "next"
    },
    "pinch_hold": {
      "type": "media", "action": "play_pause"
    }
  }
}

One JSON file per app, checked against a schema at startup. No gesture can send, post or pay on its own.

05Voice to action

Say it once. Watch every step.

Make a fist or say “Sigil”, then ask. The request becomes a short plan of visible steps, and the plan stops before anything leaves your phone.

The Control Sphere listens on the left. The voice leaves it as a wave that settles into a line of light through four glass nodes: open Messages, find Ana, write the note, send.

  1. 01 · Ask

    Say it once.

    “Tell Ana I am running ten minutes late.” Speech becomes text on the phone.

  2. 02 · Plan

    Watch every step.

    The request becomes a chain of steps. Each node lights only when its step really completes.

  3. 03 · Confirm

    Nothing sends without your yes.

    Before Send, the chain stops and asks. A pinch, a touch or “yes” continue, and 8 seconds of silence cancel.

Try the chain

Simulated here

“Sigil, send Ana on WhatsApp that I am on my way.”

    {$store.chain.steps.map((step, i) => (
  1. {step.num} {step.label}
    2. ))}

Ready. Nothing runs until you ask.

A simulation in your browser. On the phone, the same plan runs inside WhatsApp, one visible step at a time.

06On device

Seen by your phone. Kept by your phone.

Hand tracking, speech recognition and the language model run on the phone. No account, no server, no recording.

A phone holds a glowing neural sphere. Camera frames from the left and a voice wave from the right flow in and stay inside. The cloud sits outside, marked not needed.
FIG. 06Camera frames and voice flow in and stay in. The cloud is not needed.

  • Frames stay in memory

    Camera frames are processed and dropped. Only 42 numbers per frame, the points of your hand, reach the interface.

  • The wake word stays local

    The keyword spotter runs on the phone and keeps discarding the audio that comes before the word.

  • No account, no analytics

    Nothing on your screen is uploaded, logged or counted. Tuning data stays on the phone.

  • The camera never hides

    A tracking chip stays on screen while the camera runs. One tap or the words “camera off” stop it.

A model sized to your phone.

Sigil picks the largest tier that fits in 35 percent of the phone’s memory. Models download only with your consent, on Wi-Fi by default, and are verified before use.

8 GB

Sigil allows itself 2.8 GB

Model tiers by phone memory
TierWhat runsDownloadPeak memory
Rules onlyFrom 3 GBHand tracking, the system speech recognizer and a rule parser for English and Portugueseabout 8 MBabout 60 MB
LiteFrom 6 GBHand tracking, Whisper base and Qwen2.5 1.5Babout 1.07 GBabout 1.4 GB
FullFrom 8 GBFits this phoneHand tracking, Whisper small and Qwen2.5 3Babout 2.15 GBabout 2.6 GB

07Platforms

Honest about the platform.

Controlling other apps is possible only on Android. On iPhone, Sigil works inside its own screens and through Shortcuts, and says so.

Android

Full control.

Through an Accessibility service you turn on yourself, Sigil taps, swipes, scrolls, types and goes back, home or to recent apps in any app. The cursor is drawn over every app, and the camera keeps running with a visible notification you can stop.

  • Declared as an accessibility tool for hands free control
  • Turned on by you, in Settings, Accessibility
  • Acts only where you point and only when you ask

iOS

Inside Sigil, and through Shortcuts.

iOS does not let an app read, tap or draw over other apps, or use the camera in the background. On iPhone, Sigil tracks your hand inside its own screens, opens apps and searches through links, runs your Shortcuts and offers its actions as App Intents to the Shortcuts app and the Action button.

  • Gestures, cursor and sphere inside Sigil
  • Voice plans through links and Shortcuts
  • Steps iOS cannot run are reported as failed, never faked
What each platform allows
CapabilityAndroidiOS
Cursor and Control SphereOver every appInside Sigil
Gestures in other appsYes, through AccessibilityNo
Voice plansStep by step, in any appThrough links and Shortcuts
Camera with other apps openYes, with a visible notificationNo
Models on the phoneYesYes

08Follow the build

Built in the open.

The Android preview already follows your hand, taps, listens and understands on the phone. Next we tune tracking on more phones and get ready for the stores.

  1. M1 · Done

    Skeleton

    Gesture engine with tests, app profiles, planner, confirmation policy, plan executor and the Android control module.

  2. M2 · Done

    Live hands

    MediaPipe hand tracking on the camera, calibrated for each person.

  3. M3 · Done

    Android end to end

    Gestures and plans acting in WhatsApp, YouTube, TikTok and Chrome on real phones.

  4. M4 · Done

    On device language

    Speech and a tool calling model on the phone, sized by its memory.

  5. M5 · Next

    Stores

    App Intents and Shortcuts on iOS, and the accessibility review on Google Play.