Playing Minecraft by voice with Talon

February 12, 2019

Edit 02/2021: this article was written about what is now a legacy version of Talon. The concepts remain the same but the scripts probably won’t work on the latest version of Talon. Visit https://talonvoice.com/ for the most recent documentation.

In times past I was an avid console gamer. Those days ended about a decade ago when my hands became too weak to operate a controller. I began to be interested in PC games that had simple controls and mechanics. At one point, with the help of my iPhone, I was able to access enough keys to play the beloved Minecraft. Several years ago I lost the ability to play that game as well. However, I’m back to playing Minecraft again—this time by voice with Talon.

Full keyboard access

What makes playing Minecraft possible is having full access to the keyboard. I’m able to control a mouse. And I can press the left click button (sometimes a second button, depending on my setup). After turning on auto-jump, a built-in Minecraft feature, I depend on Talon to handle all other functionality. Even without customization for Minecraft, including the basic_keys.py example script is enough to give you access to your inventory as well as the ability to place blocks. There are two important functions that require more nuanced behavior than simple keypresses—walking and digging. When I’m using my two-button setup, I can use one button for walking forward and the other for digging. But when I’m using my one-button setup, I need to use Talon for one of those. That’s where customization comes in.

Custom Talon scripts for Minecraft

I explained in my last article about Talon how Talon can be scripted to perform custom actions. Scripting Minecraft controls required me to dig in a little deeper, going beyond keypresses by programming macro-like sequences. For example, digging. I needed the ability to hold down the attack key for specific amount of time, depending on the block I’m trying to break. I came up with this format:

1"attack AMOUNT_OF_TIME"
"attack AMOUNT_OF_TIME"

I settled on using half-seconds as the time increment. With the command added to the keymap, it looks like this:

1ctx.keymap({
2    'attack (1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9)': hold_key
3})
ctx.keymap({
    'attack (1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9)': hold_key
})

To use this command, I speak the word “attack” followed by a number one through nine. When Talon recognizes that command, it passes my dictation to the hold_key function. The hold_key function looks like this:

 1from talon import ctrl
 2from ..utils import parse_words_as_integer
 3 
 4def hold_key(m):
 5    keymap = {
 6        'attack': 'k',
 7        'eat': 'u'
 8    }
 9    half_seconds = parse_words_as_integer(m._words[1:])
10    if half_seconds != None and half_seconds > 0 and half_seconds < 10:
11        microseconds = half_seconds * 500000
12        key_to_press = keymap[m._words[0]]
13        ctrl.key_press(key_to_press, hold=microseconds)
from talon import ctrl
from ..utils import parse_words_as_integer

def hold_key(m):
    keymap = {
        'attack': 'k',
        'eat': 'u'
    }
    half_seconds = parse_words_as_integer(m._words[1:])
    if half_seconds != None and half_seconds > 0 and half_seconds < 10:
        microseconds = half_seconds * 500000
        key_to_press = keymap[m._words[0]]
        ctrl.key_press(key_to_press, hold=microseconds)

There’s kind of a lot going on here, but essentially what’s happening is that it’s figuring out which key I want to press and then it’s holding it for the specified number of half-seconds. The parse_words_as_integer function comes from the community utils.py script. It’s taking a spelled out number from my dictation (e.g., “four”) and turning it into an actual number (e.g., “4”)—an integer type—that I can use to set the hold time. It’s a handy command, but when I’m doing a lot of digging, it gets repetitive. That’s okay, because I have another option. I made a second command for attack that simply holds down the key until I tell it to stop.

1def start_attack(m):
2    ctrl.key_press('k', down=True)
3 
4def stop_attack(m):
5    ctrl.key_press('k', up=True)
def start_attack(m):
    ctrl.key_press('k', down=True)

def stop_attack(m):
    ctrl.key_press('k', up=True)

Added to the keymap, it looks like this:

1ctx.keymap({
2    'attack (1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9)': hold_key,
3    'hitter': start_attack,
4    'stop': stop_attack,
5})
ctx.keymap({
    'attack (1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9)': hold_key,
    'hitter': start_attack,
    'stop': stop_attack,
})

If you’re interested in more, you can see my full Minecraft script as well as the rest of my user scripts on GitHub.

Endless possibilities

After being limited to mouse-only and turn-based games for a while now, I’m excited by the possibilities Talon brings for gaming. I can imagine writing custom scripts for something like World of Warcraft—speak the phrase “go ham,” and my character starts firing off its attack rotation. Talon could also help with in-game text chat, something that hasn’t been feasible for me before. I honestly can’t say whether I’m having more fun writing scripts for Talon or using them to write, code, and play games. #nerd #winning