Basics of Input Processing

Handling Input

Let's introduce key and mouse input. We'll start our interactive-drawing lab with cargo new --bin interactive-drawing: 

[package]
name = "interactive-drawing"
version = "0.1.0"
authors = ["Joseph C. Osborn <joseph.osborn@pomona.edu>"]
edition = "2021"

[dependencies]
env_logger = "0.10"
image = "0.24"
log = "0.4"
pollster = "0.3"
wgpu = "0.17"
winit = "0.28"
imageproc = "0.23"

We'll start from the textured triangle code, so copy that over into your new src/ folder. I think it would be nice if we could modify load_texture slightly to also return the loaded image alongside the texture: 

fn load_texture(
    path: impl AsRef<std::path::Path>,
    label: Option<&str>,
    device: &wgpu::Device,
    queue: &wgpu::Queue,
) -> Result<(wgpu::Texture, image::RgbaImage), image::ImageError> {
    // ^^^ new return type
    // ... everything else is the same until the final line:
    Ok((texture,img))
}

// And at the call site...
// let (tex_47, mut img_47) = load_texture("content/47.png", Some("47 image"), &device, &queue).expect("Couldn't load 47 img");

Before, we handled three kinds of events: Window resize events, window redraw events, and window close events. To support interactivity, we'll add mouse and keyboard events this time around, and let the user scribble onto the texture.

It's generally good to put all the input-handling code together in one place, and to keep that separate from the drawing code. For now, we'll use the input loop to organize our game code.

First, let's store the currently held keys and previously held keys. These will be the main structures we query to determine what actions to trigger: for durative actions like arrow key movement, we might want to just check whether the right arrow is presently held, whereas for discrete actions like jumping we probably want to be sure that the space bar is pressed right now but wasn't pressed before.

winit defines 255 "virtual" key codes, which map in a cross-platform and cross-keyboard-layout way. We'll have two 255-bool arrays to track key states, so there's no need to do a search to see if a key is up or down. Another good approach could be two vecs, since the number of keys held down simultaneously is likely to be very small. We'll make a new module called input to track keyboard and mouse states over time (go ahead and grab input.rs and put it next to your main.rs). Then you can use it like so: 

// At the top of the file somewhere, maybe after the use statements

mod input;

// Then just before we run the event loop...

// let mut input = input::Input::default();

Then, we need to extend the loop in our winit callback to handle the two parts of the input handling lifecycle: handling events in the stream, and noticing the end of one event stream and start of the next one.

First, we want to tell our input handler that prev_keys should be reset to now_keys on Event::MainEventsCleared in our big match statement on the current winit event. 

match event {
    // ...
    Event::MainEventsCleared => {
        // Leave now_keys alone, but copy over all changed keys
        input.next_frame();
        // ... all the stuff that was there before...
    },
    // ...
}

Next, we need to handle keyboard and mouse events. This will be in the same big match: 

match event {
    // ...
    // WindowEvent->KeyboardInput: Keyboard input!
    Event::WindowEvent {
        // Note this deeply nested pattern match
        event: WindowEvent::KeyboardInput {
            input:key_ev,
            ..
        },
        ..
    } => {
        input.handle_key_event(key_ev);
    },
    Event::WindowEvent {
        event: WindowEvent::MouseInput { state, button, .. },
        ..
    } => {
        input.handle_mouse_button(state, button);
    }
    Event::WindowEvent {
        event: WindowEvent::CursorMoved { position, .. },
        ..
    } => {
        input.handle_mouse_move(position);
    }
    _ => (),
}

Coloring Time

Let's do a little exercise. First, add a couple of fields just before the event loop: 

let mut color = image::Rgba([255,0,0,255]);
let mut brush_size = 10_i32;

Now, insert this code into the event handler for MainEventsCleared: 

Event::MainEventsCleared => {
    // Your turn: Use the number keys 1-3 to change the color...
    // (1)
    // <YOUR CODE HERE>
    // And use the numbers 9 and 0 to change the brush size:
    if input.is_key_down(winit::event::VirtualKeyCode::Key9) {
        brush_size = (brush_size-1).clamp(1, 50);
    } else if input.is_key_down(winit::event::VirtualKeyCode::Key0) {
        brush_size = (brush_size+1).clamp(1, 50);
    }
    // Here's how we'll splatter paint on the 47 image:
    if input.is_mouse_down(winit::event::MouseButton::Left) {
        let mouse_pos = input.mouse_pos();
        // (2)
        let (mouse_x_norm, mouse_y_norm) = ((mouse_pos.x / config.width as f64),
                                            (mouse_pos.y / config.height as f64));
        imageproc::drawing::draw_filled_circle_mut(
            &mut img_47,
            ((mouse_x_norm * (img_47_w as f64)) as i32,
             (mouse_y_norm * (img_47_h as f64)) as i32),
            brush_size,
            color);
        // We've modified the image in memory---now to update the texture!
        // This queues up a texture copy for later, copying the image data.
        queue.write_texture(
            tex_47.as_image_copy(),
            &img_47,
            wgpu::ImageDataLayout {
                offset: 0,
                bytes_per_row: Some(4 * img_47_w),
                rows_per_image: Some(img_47_h),
            },
            wgpu::Extent3d {
                width:img_47_w,
                height:img_47_h,
                depth_or_array_layers: 1,
            },
        );
    }
    // Remember this from before?
    input.next_frame();

    // ... All the 3d drawing code/render pipeline/queue/frame stuff goes here ...

    // (3)
    // And we have to tell the window to redraw!
    window.request_redraw();
},

Looking at the code above, try to answer these questions to complete your paint program:

  1. At point (1) in the code, how can we check for the number keys 1-3? And how can we update the color field with a new color value?
  2. At point (2), what is the purpose of the division and (in the next statement) the multiplication of the mouse coordinates?
  3. At point (3), try removing the request_redraw() call. What happens?

What's next?

We didn't talk about text rendering at all. It's easy to do an easy version (as the imageproc crate does) and very, very hard to do a correct version—some crates out there will help you (e.g. rusttype, fontdue).

Bitmapped fonts are straightforward to lay out left to right on a fixed grid using the same basic idea as sprite blitting (which is our next topic), and even word-wrapping isn't so bad if you know in advance how many letters your word is and how much horizontal space you have. But handling everything from right-to-left text to Unicode to text formatting in a reasonable way is a huge can of worms.

It's common for older games to include in their scripts both text and "formatting commands" that can do things like start a new line or dialog box, change the current text color, change the speaker's portrait image, set or check plot flags, and so on; so your text layout routine takes on the flavor of a programming language interpreter! If you're interested in learning more about this, get in touch.

Lab: Interactive Drawing

Extend the interactive drawing example from above however you like. For full credit, do two of the following:

  1. Draw three distinct types of shapes interactively by defining new "tools", e.g. by dragging with the mouse you can stretch out a rectangle. Some ideas:
    1. Filled rectangles
    2. Outlined rectangles
    3. Outlined polygons
    4. Bresenham-lines
    5. Circles
    6. Text
  2. Draw with partial transparency (check out the Blend type from imageproc), with keys to increase and decrease the alpha value of the current color. There are some hints on Zulip about this.
  3. Something else, let me know what you have in mind!