About this course
Human-Machine Interaction (HMI) bridges cognitive science, design, and AI to produce interfaces that are effective, efficient, and inclusive. Students move from foundational perception and cognition theory through structured UX design processes, multimodal input channels, and accessibility standards, concluding with AI-augmented interaction patterns such as adaptive UIs and conversational agents. Every concept is anchored in a semester-long design-and-evaluation project.
Course format. Thirteen weeks, four contact hours each: a two-hour lecture (concepts and theory) and a two-hour practice session. The course is project-based; teams carry one running project end to end and present it three times, in weeks 5, 8, and 13.
What you will build
Teams design, prototype, and evaluate one human-machine interface for a real system or well-defined domain problem. The interface evolves across 13 weeks from user-research synthesis through low-fidelity concept sketches into a high-fidelity Figma prototype with at least one implemented multimodal input channel (voice, gesture, or gaze via Web Speech API), an AI-augmented layer (adaptive recommendation or conversational UI via OpenAI API), WCAG 2.2 Level AA compliance, and a final usability evaluation report backed by two rounds of moderated testing and a designed A/B test.
Expected outcomes
- Apply human information-processing theory, mental models, Norman's gulfs, affordances, Fitts' Law, GOMS models, and Gestalt principles to analyze and predict interaction behavior across interface designs
- Conduct contextual inquiries, user interviews, and card-sorting studies; synthesize findings into personas, journey maps, and information-architecture specifications using Miro and Optimal Workshop
- Design and iterate UX artifacts from low-fidelity wireframes to interactive Figma prototypes, building annotated design-system component libraries that conform to WCAG 2.2 Level AA
- Plan and execute two rounds of moderated usability testing with SUS scoring via Lookback, remote unmoderated sessions via Maze, and drive redesign from quantitative evidence analyzed with Python
- Integrate at least one multimodal input channel via the Web Speech API and an AI-augmented layer into a working prototype, evaluating each with structured testing and a final usability report
- Instrument product analytics with PostHog, design and analyze a controlled A/B test, and translate quantitative usage data into evidence-based design decisions
Key topics
- Cognitive models of interaction
- UX design and design thinking
- Multimodal interfaces
- Accessibility and inclusive design
- AI-augmented interaction
- Usability testing and evaluation
Theoretical foundations
The concepts and results this course rests on.
- Human information-processing theory: sensory, working, and long-term memory; attention bottlenecks and perceptual chunking as constraints on interface design
- Mental models and Norman's gulfs of execution and evaluation: conceptual model mismatch as the root cause of usability failure
- Affordances, signifiers, mappings, and action-feedback loops: perceived affordances and natural mappings in physical and digital interfaces
- Fitts' Law: target acquisition time, size-distance trade-off, throughput in pointing tasks, and the Steering Law for constrained-path navigation
- GOMS models and the Keystroke-Level Model: Goals, Operators, Methods, and Selection rules for rapid task-time prediction and design comparison
- Gestalt principles for visual layout: proximity, similarity, continuity, closure, and figure-ground segregation as perceptual organizing rules
- Multimodal fusion theory: touch, voice, gesture, and gaze as input channels; cross-modal conflict resolution and graceful fallback strategies
- WCAG 2.2 POUR framework: perceivable, operable, understandable, and robust success criteria at levels A, AA, and AAA
- A/B testing theory: hypothesis formulation, statistical significance, minimum detectable effect, novelty effects, and controlled experiment design for UX decisions
- Adaptive and personalized systems: user modeling, collaborative filtering, context sensing, and the privacy-utility trade-off in interface personalization
Prerequisites
This is a Year-3 course. It assumes the mandatory CS core: data structures and algorithms, operating systems, computer networks, databases, software engineering, and the core mathematics (linear algebra, probability and statistics, calculus, discrete mathematics). It additionally requires the specific prior courses listed below.
Course-specific prerequisites:
- Software Engineering
- Human-Computer Interfaces (HCI/UI)
- Probability and Statistics
Weekly schedule 13 weeks · lecture + practice
Foundations
Wk 1
Cognitive Foundations of HCI
LectureIntroduce the human information-processing model: sensory perception, working memory capacity, long-term memory retrieval, attention bottlenecks, and perceptual chunking as core constraints on interface design.
PracticeMiro: collaboratively map cognitive-load violations and attention failures on an existing public interface using a structured affinity diagram.
ProjectProject domain and target user group are chosen; domain background research is begun.
Wk 2
Mental Models, Affordances, and Signifiers
LectureCover Norman's action cycle, the gulfs of execution and evaluation, conceptual model mismatch, affordances, signifiers, natural mappings, and action-feedback loops as the vocabulary of interaction analysis.
PracticeFigma: annotate a real interface identifying affordance breakdowns and signifier failures, then sketch a redesigned version that closes the gulf of evaluation.
ProjectCognitive walkthrough of the chosen domain's current interface is produced and findings are documented.
Wk 3
Fitts' Law, GOMS, and Task Modeling
LecturePresent Fitts' Law for target acquisition time, the size-distance trade-off, throughput, and the Steering Law; then GOMS Goals, Operators, Methods, and Selection rules and the Keystroke-Level Model for rapid task-time prediction.
PracticeOptimal Workshop: run a card-sorting study with classmates to derive the information architecture for the project domain; analyze dendrograms and agreement matrices.
ProjectUser research plan is finalized; at least three contextual inquiries and user interviews are conducted and synthesized.
Design
Wk 4
Gestalt Principles and UX Design Process
LectureExamine the Gestalt laws of proximity, similarity, continuity, closure, and figure-ground segregation applied to visual layout; situate them within design thinking as an iterative empathize-define-ideate-prototype-test cycle.
PracticeMiro: run a structured ideation workshop producing project personas, user journey maps, and three competing low-fidelity concept sketches; use dot-voting to select the primary direction.
ProjectPersonas, journey maps, and three low-fidelity concept sketches are delivered; one concept is selected with documented rationale.
Wk 5
Specification and Wireframes MilestonePresentation
LectureDiscuss prototyping fidelity levels, interaction specification techniques, and design-system foundations: type scales, color roles, spacing tokens, and component anatomy.
PracticeTeams present their project specification: user-research synthesis, chosen concept, Figma low-fidelity wireframes, and proposed evaluation plan; peers give structured critique.
ProjectApproved specification with low-fidelity Figma wireframes, research synthesis, and evaluation plan is delivered.
Multimodal and Accessibility
Wk 6
Multimodal Interfaces: Voice, Gesture, and Gaze
LectureExamine multimodal fusion theory: touch, voice, gesture, and gaze as parallel input channels; cross-modal conflict resolution, graceful fallback strategies, and voice interface design heuristics for conversational patterns.
PracticeWeb Speech API: add a voice-command layer to a browser-based prototype implementation; evaluate recognition accuracy and interaction flow with a structured walkthrough.
ProjectAt least one multimodal input channel (voice, gesture, or gaze) is designed and integrated into the high-fidelity prototype.
Wk 7
Accessibility and WCAG 2.2
LectureExamine the WCAG 2.2 POUR framework in detail: perceivable, operable, understandable, and robust success criteria at levels A, AA, and AAA; semantic HTML, ARIA roles and properties, and remediation patterns for common barriers.
PracticeWAVE: audit the project prototype for Level AA violations using visual feedback overlays; document each barrier by criterion, severity, and remediation; retest after fixes.
ProjectProject prototype passes automated WCAG 2.2 Level AA audit; manual screen-reader testing results are documented.
Wk 8
Interim Demo MilestonePresentation
LectureCover moderated usability testing methodology: participant recruitment, think-aloud protocol, task design, session facilitation, severity rating, and how to structure a usability findings report.
PracticeTeams demonstrate the high-fidelity prototype with the multimodal layer and WCAG compliance evidence in a live moderated walkthrough; facilitators practice note-taking and probing techniques.
ProjectHigh-fidelity prototype with multimodal input channel and WCAG 2.2 Level AA evidence is demonstrated live.
Evaluation and AI-Augmented Interaction
Wk 9
Usability Testing and SUS Evaluation
LectureCover SUS scoring and benchmarking, heuristic evaluation with Nielsen's ten heuristics, remote vs. moderated test trade-offs, and converting severity-ranked findings into a prioritized redesign backlog.
PracticeLookback: conduct a recorded moderated usability session on the interim prototype with two recruited participants; score SUS; identify and rank the top five usability problems by severity.
ProjectFirst round of moderated usability testing is complete; SUS scores are computed and a redesign backlog prioritized by severity is produced.
Wk 10
A/B Testing and Product Analytics
LecturePresent A/B testing theory: hypothesis formulation, statistical significance, minimum detectable effect, sample-size calculation, novelty effects, and instrumentation strategies for measuring the impact of design changes in production.
PracticePostHog: instrument custom analytics events in the prototype; configure a feature flag for a design A/B test; analyze mock interaction data to identify usage patterns and validate the test setup.
ProjectA/B test design is complete with a written hypothesis, sample-size calculation, instrumented prototype, and analysis plan.
Wk 11
Adaptive Interfaces and User Modeling
LectureIntroduce adaptive and personalized systems: user modeling strategies, collaborative filtering for UI recommendations, context sensing, and the privacy-utility trade-off; design patterns for AI-powered adaptive features and conversational UI layers.
PracticePython: compute descriptive statistics and significance tests on SUS scores and task-success rates; generate a usability metrics report comparing pre- and post-redesign results.
ProjectProject includes a demonstrable adaptive recommendation or conversational UI layer integrated via the OpenAI API, with documented evaluation of its effect on task completion.
Wk 12
Remote Testing and Final Iteration
LectureCover remote and unmoderated usability testing methods; strategies for iterating from two rounds of evidence; and synthesizing qualitative and quantitative findings into a final design rationale.
PracticeMaze: configure and run a remote unmoderated test of the revised prototype; collect task-success, time-on-task, and drop-off data; compare results to round-one Lookback findings.
ProjectSecond round of usability testing is complete; final design revisions are implemented and documented with before-and-after evidence.
Capstone
Wk 13
Final Demo and DefensePresentation
LectureSynthesize HMI theory, design methodology, and AI-augmented interaction; review how cognitive models, accessibility standards, and evaluation evidence converged into each team's final design decisions.
PracticeTeams present the finished interface in an oral defense covering user research, design rationale, multimodal input channel, AI-augmented features, WCAG 2.2 compliance, and usability evidence across both test rounds.
ProjectFinal human-machine interface is delivered with design system, usability evaluation report, A/B test analysis, and WCAG 2.2 Level AA documentation.
AI tools in this course.
Students use AI assistants to generate wireframe descriptions, accessibility remediation suggestions, and usability test analysis summaries. They prompt tools to draft dialogue flows for conversational UI, suggest adaptive-interface rules from user-behavior data, and generate A/B test hypotheses from analytics patterns. AI helps create design system documentation and component specifications from Figma prototypes, and supports rapid iteration on prototype content. Because AI-generated UX copy and interaction flows can introduce inconsistency or violate established design principles, students validate every AI-assisted artifact against the cognitive models, usability heuristics, and accessibility standards covered in lecture before incorporating it into the project.
Student project
Teams design, prototype, and evaluate one human-machine interface for a real system or a well-defined domain problem. The interface evolves across 13 weeks from user-research synthesis through low-fidelity concept sketches into a high-fidelity Figma prototype with at least one implemented multimodal input channel (voice, gesture, or gaze via Web Speech API), an AI-augmented layer (adaptive recommendation or conversational UI via OpenAI API), WCAG 2.2 Level AA compliance, and a final usability evaluation report backed by two rounds of moderated testing and a designed A/B test.
Requirements
- Build a working prototype, not a set of disconnected screens.
- Be original: a new interface that solves a real user problem, not a reskin of an existing tutorial product.
- Show real depth: real user research, real usability testing with at least five participants per round, and accessibility evidence from both automated and manual testing.
- Carry one running project from user research through final evaluation across the whole term.
- Work in a team of three or four and defend the design at each of the three presentations (weeks 5, 8, and 13).
Example projects
Accessibility-first task management appVoice-controlled smart home dashboardAdaptive e-learning interfaceConversational medical intake assistantGesture-driven data visualizationAI-personalized news readerMultimodal navigation interface for driversInclusive public kiosk redesign
Assessment & grading
Grading is project-based, with no written exam. Teams of three or four present one running project three times.
| Component | What it covers | Weight |
|---|---|---|
| Project · Specification | Presentation 1 (week 5): user research synthesis, chosen concept, wireframes, and evaluation plan | 20% |
| Project · Interim | Presentation 2 (week 8): high-fidelity prototype with multimodal layer and WCAG compliance demonstrated live | 30% |
| Project · Final | Presentation 3 (week 13): full design with AI-augmented features, two usability rounds, and A/B test analysis | 50% |
Tools & platforms
- Figma: design, prototype, and maintain the component library and annotated design system
- Miro: run collaborative ideation workshops, affinity diagrams, and journey mapping sessions
- Optimal Workshop: conduct card-sorting and tree-testing studies to derive and validate information architecture
- WAVE: audit interfaces for WCAG 2.2 Level AA violations with visual feedback overlays
- SUS (System Usability Scale): quantify perceived usability across moderated and remote test sessions
- Lookback: record and replay moderated remote usability sessions with screen and audio capture
- Web Speech API: implement voice recognition input in browser-based prototype implementations
- PostHog: instrument interaction analytics and configure feature-flag A/B tests in web prototypes
- Python: analyze SUS scores, task-success rates, and A/B test results with descriptive and inferential statistics
- Maze: run remote unmoderated usability tests and collect task-success, time-on-task, and drop-off metrics
Free online courses
Existing free, video-based courses this course can build on, for self-study or as a teaching basis.
- CourseraHuman-Computer Interaction
- CourseraInteraction Design Specialization
- YouTubeUdacity UX Design for Mobile Developers
In Hebrew · בעברית
- Campus ILפלטפורמת Campus IL
Primary literature
Seminal works for advanced study.
References
Books and resources link to an online or publisher page.
- TextbookThe Design of Everyday Things
- TextbookDesigning the User Experience
- TextbookMeasuring the User Experience
- StandardWeb Content Accessibility Guidelines (WCAG) 2.2
- PaperGuidelines for Human-AI Interaction
- DocumentationFigma Resource Library
Role in each concentration
| Concentration | Role |
|---|---|
| Intelligent Software Systems | Core · Semester 3 |
| Networking & Cyber Security | Elective |
| AI & Robotics | Elective |
| AI and Quantum Computing for Finance | Elective |
| Immersive Systems & Game Development | Elective |
| Defense Technologies & Autonomous Systems | Elective |