On first glance, the toaster looks like a toaster of the classic 1950s era, complete with polished aluminum sides. Inside, however, is a completely different story. In an external logic box 1 is a microcontroller that processes the sensor input and generates appropriate responses.
Sensors include a motion sensor to detect a toast-hungry person; photoresistors to detect the presence of bread to be toasted; and a temperature sensor to measure the energy transmitted to the bread while toasting. For the user interface, there is speech recognition and voice response. The speech recognition allows the user to simply speak commands to the toaster. The voice response module provides friendly feedback to the user based upon the user's selection.
To ensure the safety of the toast connoisseur, the power lines in the toaster are properly insulated. Additionally, to ensure the safety of the toaster, the microcontroller will be shielded from the line current switch by external circuitry.
The solution we propose is to use a voice activated user interface, where the food preparing device is controlled by commands spoken by the Computer Scientist. In turn, the food preparing device will also respond to the Computer Scientist with prerecorded voice clips. The typical Computer Scientist conducts most of his or her correspondence via electronic mail, so the vocal cords are hardly stressed. The voice user interface is the best.
As a prototype for this new user interface, we have designed and built one of the simpler food preparing devices under the new paradigm: we have constructed a Talking Toaster.
Another big win from modularity is the debugging of the final assembled system. Given that each component had been tested and debugged on its own, if the final assembly did not work, we could concentrate our debugging effort on the interconnection of each component. Further, we could add components one by one to the final assembly, reducing the number of interconnections that needed to be checked should something go wrong.
While designing and testing these systems, we noticed that the voice clips played back sounded noisy. Through some experiments with an oscilloscope, we determined that both the power supply and evaluation board (which housed the microcontroller during initial prototyping) were electronically noisy. To deaden this noise, bypass filter capacitors (1000mF) were placed from power to ground near the voice response and speech recognition chips.
Voice Response
The voice response subsystem was implemented with an Information Storage
Devices' ISD1000A analog memory chip. This chip allowed for up to 20
seconds of response to be stored for playback. Further, the storage was
nonvolatile, so no battery backup was necessary.
There were two aspects of the voice response system. The first was playing responses back at the appropriate times. The second was recording the responses to be used in the toaster. Because the responses are not to be changed by the user, there is no recording capability in the toaster's circuit. This greatly simplifies the circuit, mainly because the play/record pin on the voice chip may be permanently tied to play. The other alternative, where the microcontroller sets and clears the play/record pin, is troublesome during debugging because the microcontroller may hang and accidentally assert the record signal and erase the voice chip.
The play-only circuit using the ISD1000A chip may be found in the upper right corner of the Main Controller Logic Board schematic in Appendix A. To specify which voice clip to play, the microcontroller sends an 8-bit address out on its port B. To play the voice clip, the microcontroller pulses the ISD1000A's chip enable (CE) signal using port C[7]. To signal that the end of the message has been reached, the ISD1000A pulses the end of message pin (EOM) for 16 milliseconds.
The recording circuit is not pictured, but is not any more complex than what is shown. The only changes are the addition of an electret microphone and some passive components on the MIC line, and the microcontroller driving the play/record (P/R) pin.
To experiment with the voice response system, an assembly code module was written that interfaces the microcontroller to the ISD1000A. Later in the construction of the toaster, this module became quite valuable as an instrument with which to record the voice responses. To aid in debugging, the ISD1000A driver module was written using the same microcontroller ports as would be used in the final design. That is, no wires had to be moved, etc., when first debugging the entire design, and then verifying just the voice response system.
Speech Recognition
The speech recognition subsystem was built using Hualon Microelectronics
Corporation's HM2007L speech recognition chip. This chip allowed for
words to be recognized from a vocabulary of up to 40 1-second long words.
The vocabulary was stored on an external 8K SRAM. This SRAM was not
powered by a battery, so the vocabulary was lost when the toaster was
unplugged. While this may not be ideal for the Computer Scientist, this
removed a design constraint that otherwise would have hampered the
project.
To use the speech recognition system of the HM2007L, the user must train their voice prints on the chip. In the current version of TOS, the user is instructed on how to do this when the toaster is first plugged in. For each word that is to be recognized, the microcontroller asks the user to speak that word. Because the user may say the word differently (i.e., with slightly different inflections, etc.), the user is asked to say the word more than once (usually three times). For each time the user says the word, the HM2007L integrates this word into a neural network (this network is stored in the off-chip SRAM). Later, in recognition mode, the HM2007L tries to match the spoken word against other words in its neural net. If a match is made, the index of that word in the vocabulary is returned. If no match is found, or if the user spoke too quickly or too slowly, an appropriate error code is returned.
Thus, the HM2007L does not recognize a spoken word as an actual word, but rather as sounding like a word that it knows about. The HM2007L has no a priori knowledge of what the word 'toast' should sound like.
The implementation of the speech recognition system was by far the most difficult engineering feat in the entire project. The chip came with a data sheet, but the information contained therein was in many instances unclear and even incorrect. The distributor of the chips had no experience with using the chips in CPU mode 3, so we were on our own. Fortunately, after two weeks of intensive experimenting and designing, we had an understanding of the basic program flow required to train and recognize words.
The HM2007L is a 52-pin PLCC chip, with a 48-pin DIP counterpart, the HM2007P, and a bare-pad version, simply HM2007. The HM2007L has 4 I/O ports, a microphone system, and several control pins. To communicate with the 8K SRAM, there is a 13-bit address bus and a 8-bit data bus (two of the four I/O ports), as well as a memory read/write pin and a memory enable pin. To communicate with the microcontroller, there is a 4-bit wide K-bus, used for passing data to and from the HM2007L, and a 3-bit S-bus, used for sending commands to the HM2007L (mainly commands to control the direction and meaning of the K-bus). Because we had only two of these chips, and no more were expected to be available from the manufacturer until after the end of the quarter, we took great care in protecting our chips from possible shorts. The S-bus and K-bus were connected to the microcontroller's C port via 10kOhm resistors. If both chips tried to drive the lines at different logic values, at least there would be no direct short through the chips.
Like the ISD1000A, a separate assembly code module was written to experiment and debug the speech recognition chip. And like its counterpart with the ISD1000A, this module was invaluable in debugging the full prototype.
The next issue was modularity. From the start, easy access to the toaster's working parts had to be engineered into the design to facilitate easy debugging and experimentation. Different positions for the temperature and bread sensors were tried to seek the optimal position. The motor used to raise and lower the bread platform needed to sit in exactly the right position.
To aid in this modularity issue, a quick-release toaster cover was fabricated. This involved using some attachment screws on the toaster handles and removing excess metal from the chrome cover. The feet of the toaster were fixed to the toaster directly, as opposed to being fixed to the chrome cover. The toaster, in turn, was secured to the external logic box by four thumb screws. In total, these modifications allowed us to debug the entire design both with and without the toaster, depending upon what was easier at the time.
Temperature Sensor
The first sensor installed was the temperature
sensor. The temperature sensor sampled the current temperature inside the
toaster, which was read through the A/D port on the microcontroller. When
a threshold temperature was reached (determined by the toast setting the
user had indicated: light, medium, or dark), the microcontroller would
raise the bread platform.
The temperature sensing device we used was Analog Device's AD590J, a temperature transducer. The particular part we used was an integrated circuit mounted in a metal can (AD590JH). The AD590J reported the temperature by outputting a current in mA equal to the temperature in degrees Kelvin 4. Thus, at room temperature (298 K), the AD590J would output 298mA of current.
The operating range of the AD590J is between 218K and 423K (-55 deg and 150 deg C). While this is a wide range extending into high temperatures, the interior of the toaster raises far hotter than this. Through experimenting, we found the best position for the sensor, one where it did not reach its upper limit before the toast was done, was just beneath the toaster.
Once the temperature sensor was mounted, and toasting tests were begun, we began to notice erratic behavior from the AD590J. As it turned out, the A/D port on the microcontroller was experiencing terrible noise every 32ms, just when the servo motor's controlling signal was reset. The noise manifested itself as ringing in the lower two or three bits of the A/D channel. While this should not have much of an impact, it did in this case, because we had lower resolution on our temperature sensor (see below). Thus, we needed a fix. The solution: simply ignore temperature readings during this noisy period. Because the noise lasted for less than 12ms at every timer rollover, we could simply look at the microcontroller's timer and ignore the data if the counter were less than 25k.
Bread Sensor
In order to make toast, the toaster must assure that there is bread
loaded. To check the bread loaded condition, we mounted photoresistors 5 on the external logic box, looking up through the
bread slots in the toaster. Under normal conditions, light in the room
would filter through the toaster and strike the photoresistors, keeping
their resistance at some nominal level. When bread is inserted, the light
no longer makes it all the way down to the photoresistors, and their
resistance changes. Using the circuit found in Appendix A (SDS Controller board), the change in
resistance was converted into a change of voltage, which was in turn read
by the microcontroller's A/D port.
Motion Sensor
The motion sensor turned out to be easier than was
originally planned. In the final version of the toaster, a motion sensor
from a security porch light was used. The motion sensor was a remote unit
that ran on two 9v batteries. When the unit detected motion, it would
send a radio signal to a receiver in the toaster, which would close a
relay, raising a signal on the microcontroller. The microcontroller would
respond accordingly, and set a flag. The problem is that, once motion has
been detected, the relay remains closed until the motion stops, and then
for 12 seconds after that. During that time, the toaster should not be
assuming that it needs to react to the motion. Thus, a flag is set and
not cleared until the relay has been opened again.
Bread Platform
In parallel with the development of the
sensors was the work on the bread platform. After many false starts,
including a long and windy path of learning how stepper motors work, we
made the observation that a servo motor would do exactly the right job for
us. Where a stepper (or other non-servo motor) would easily be driven by
the microcontroller, there would be a need for feedback in the toaster's
up and down positions. That is, when the bread platform reached its
maximum or minimum height, the microcontroller would have to be
interrupted. This would involve another two or three I/O pins that we
could not afford. On the other hand, using a servo would eliminate the
need for this feedback. By simply sending an accurate pulse-width
modulated (PWM) signal to the motor, the shaft would turn to exactly the
right position, and hold there as long as the PWM signal was held.
SDS Controller Board
All the sensors mounted in and on the external logic box needed some way
to be powered and/or interfaced into the microcontroller. Because the
microcontroller and other ICs were wirewrapped, a separate board for the
SDS seemed most logical. Thus, we have the SDS Controller Board. A
schematic for the SDS Controller Board appears in Appendix A.
Once again recalling that modularity was a key design factor, we incorporated that into the SDS Controller Board. To interface to the Main Controller Logic Board, the SDS had an 8-conductor cable terminated with an RJ-45 connector. The Main Controller Logic Board had an RJ-45 socket wirewrapped accordingly, and the two could easily be connected and disconnected as need be.
Besides offering a convenient method of connecting the sensors to the wirewrap board, the SDS Controller Board also generated the high and low reference voltages for the microcontroller's A/D converter. The A/D converter required at least 2.5v between the low and high references, so we built a voltage divider that offered 2.6v difference. This was much more than we would have liked; the AD590J had a voltage swing of 1 volt. To make the case worse, we were interested in only a fraction of the temperature sensor's range. The difference between when light toast is done and when dark toast is done is only a few degrees. Unfortunately, due to time constraints, we could not apply any remedy for this situation, and had to suffice with a few bits of resolution on the temperature sensor.
The basic flow of logic in TOS is as follows:
Initialize
Has the user said 'toast'?
Has there been movement?
If the user asked for toast, or if there has been movement, check to see
if there is bread in the toaster. If there is no bread, ask the user to
load the toaster and try again. Else, ask the user how light (light,
medium, or dark) they would like their toast. Toast the bread, and hail
the user when its all done. Return to start.
For each subsystem (speech recognition, voice response, motor control, sensors), a separate driver module was written that tested that component. As the components were completed, their drivers were added to TOS. This allowed for TOS to grow in parallel with the design work on the rest of the toaster. This parallelism was a big win for us, as it kept both of us busy, not waiting for one another.
This prototyping method also lead to the use of the EVB's Buffalo functions for displaying information on the terminal, and for setting up interrupt service routines. When the TOS was converted to be run on the wirewrapped Main Controller Logic Board, with its embedded MC68HC11E2 microcontroller, the code had to be modified (this lead to the e2 extension for versions of TOS that are wirewrap-board specific). In particular, the functions provided by buffalo were no longer available, and had to be "stubbed out". This is what we see by the NULLFCN in TOS v1.6e2. Also, the origin of the code, data, and stack were different on the 'E2 chip. Code started at address $F800. The stack had to be explicitly loaded, so the address $0047 (inside the 256 bytes of on-chip RAM) was used. Stack grows down, so there are 71 bytes of stack space available in this configuration. For global data, the origin was selected as $0048, just above the stack.
One more change necessary when porting TOS to the 'E2 chip was loading in the interrupt service routines. This is taken care of in a strange manner on the evaluation board, through the pseudo-vectors. On the 'E2 chip, we had to explicitly program the reset and IC1 ISRs in the assembly source code.
We also learned that too much planning can be as bad as too little planning. From the start, we had tried to specify certain components, like what type of motor to use for the bread platform. By attempting to nail these components down early on, we locked ourselves into a mind set that other parts would not due. This was particularly true for the bread platform motor. We had convinced ourselves that a stepper motor with a worm gear was the way to go. In reality, this method was fundamentally flawed, and a servo motor would be by far a wiser choice. We spent at least a week and a half chasing down this blind alley, mainly because we took that long to realize that we had other options.
Another lesson that we learned has already been mentioned: planning for debugging by modularity. From the very start we had several smaller components in our design, and we could set our milestones based on the progress we made with each component. We could design, build, and debug these components each on our own, without needing to find a common time that we could share in the lab. And when it came time to mesh everything together, we had few variables to check. Each component was already known to be good, so all that's left is the few wires between the modules.
There were, of course, several little changes that we could have made, and we would have benefited from the foresight had it been available. Listed with respect to the module affected, they appear below.
Temperature Sensor
We had known all along that measuring the temperature and waiting for it
to raise above a preset threshold was not the optimal solution. However,
it gave us the easiest driver code: a simple polling loop watching the A/D
port.
Given more time, we would have liked to re-implement the sensor to measure the differential temperature from when the toast making process begins. That is, after one slice of toast has been produced, and the toaster is still hot, the toaster would toast until the temperature raised by so many degrees, not simply until it reached an absolute temperature. Another method that we could have tried is to measure heat energy transmitted (the product of temperature and time).
Bread Platform
As we have lamented before, we spent a great deal of time investigating
how a stepper motor works, when the right motor for the job was sitting
idly in someone's cupboard. If we had realized much sooner that a servo
was just what we were looking for, then maybe we could have avoided our
last hour work marathon.
SDS
Much of the time in our last-hour work marathon was spent debugging the
temperature sensor's driver software and support circuit. As things turn
out, we were using A/D channel E[0] for the temperature sensor. However,
E[0] has a 10kW resistance to it, while the other A/D ports have infinite
resistance. This impacted our temperature sensor by drawing all the
current being provided, as opposed to quietly measuring the voltage
produced across the resistor in the circuit. We really wish we had known
that E[0] was a bad idea from the start.
Speech Recognition
We're really glad that we opted for this user interface, as opposed to
push buttons or even a simple clapper-type interface. It was a major
challenge to determine how the speech recognition chip worked, but it was
well worth it. We learned a lot about safe and unsafe experimental
procedures, and how to test a chip for which there is exactly one backup.
It also gave us a great deal of practice in writing assembly code for the
microcontroller, a skill that will be valuable in the future.
Main Controller Logic Board
SDS Controller Logic Board
*************************************************
*
* Toaster Operating System
* Version 1.6e2
*
* (c) 1996 Corin Anderson and Chris Setter
*
* CSE 477, Spring Quarter, 1996
* University of Washington
*
*************************************************
********************************************************************
* Revision History
*
* 1.6e2 Fix applied to temperature sensor. At timer overflows,
* there is noise on the A/D ports, probably caused by the PWM
* signal that's driving the toaster's servo. This noise
* manifests itself as a high frequency ring in the last two
* bits of the temperature data. This is a major problem
* the difference between light and dark toast settings are
* less than 3 bits apart. The solution applied here is to
* simply ignore the temperature sensor's reading during this
* ringing period. The ringing lasts no longer than about
* 12.5 ms, or about 25K e-cycles. This solution solves the
* problem.
*
* 1.5e2 Modified to be used without the evaluation board.
* Modifications include:
* ** Globals are now at $0047
* ** Code is now at $F800
* ** RESET and IC1 interrupts are set with FDB
* statement
* ** Buffalo functions are replaced with NULLFCN
*
* 1.5 User Interface modified. No more command stuff. User
* simply says "toast" and off things go. Toaster has longer
* responses.
*
* Sensor code still being modified and tested.
*
* New voice chip vocabulary
*
* 1.4 Code added to read and respond accordingly to the bread
* and temperature sensors.
*
* Also added code to prompt the user for toasting level.
*
* Never functional. Version bumped for new UI.
*
* 1.3 Reworked speech recognition code so the toaster won't try
* to recognize its own voice (from the Say() routine).
*
* Added code to audibly prompt the user to train words. Also
* added code to display the word on the terminal. Both of
* these functions use arrays of the voice clips and words.
*
* New voice chip vocabulary
*
* 1.2 Interfaced TOS to Bread Platform servo motor. The servo
* requires a pulse width modulated (PWM) control signal to
* position the shaft angle. Thus, the output capture facility
* is used.
*
* For the safety of the toaster and of the user, there is a
* servo override button on the toaster. When this button is
* pushed, the servo is swung up to its raise position. This
* still doesn't unlock the bread platform from the motor, but
* at least this will open the contacts to the heating coils
* and free the user's bread.
*
* 1.1 Added code to disregard speech matches that have too high
* of a score. With luck, this will eliminate false positives.
* Unfortunately, this will also lead to false negatives. But
* that's usually easier to cope with.
*
* 1.0 First release of functional TOS. Includes code for speech
* recognition, voice response, and motion detection. Speech
* recognition is limited to about 5 words trained at startup.
* Voice responses are tailored for the specific command: make
* toast.
*
* 0.1 First incarnation of TOS. Does not function properly. Has
* incorrect handling of voice response chip (ISD1000A). Needs
* substantial work to be functional.
*
*********************************************************************
*********************************************************************
* Constant definitions for Speech Recognition chip
* Bus connections for Speech Recognition chip
SBUS EQU $1003 S bus is C[6:4]
KBUS EQU $1003 K bus is C[3:0]
* Bus bit mask
SMASK EQU %01110000 Middle 3 bits
KMASK EQU %00001111 Lower 4 bits
* States for speech recognition chip
srNULL EQU %00000010 Null state
srRDSTATUS EQU %00000010 Read status from K bus
srRDOUTPUT EQU %00000011 Read output buffer from K bus
srWRINPUT EQU %00000100 Write intput buffer on K bus
* Status codes
srRDYHI EQU %00000000 Ready for second (hi) nibble
srRDYVOICE EQU %00000001 Ready for voice input
srRDYCMD EQU %00000010 Ready to receive command
srRDYLO EQU %00000011 Ready for first (lo) nibble
* Command codes
srCMDRECOG EQU %00000001 Recognize word
srCMDTRAIN EQU %00000010 Train a pattern
srCMDRESULT EQU %00000100 Read result and score
srCMDRESET EQU %00000111 Clear all patterns from HM2007
* Result codes
srTOOSLOW EQU $55 Voice was too long (too slow)
srTOOFAST EQU $66 Voice was too short (too fast)
srNOTFOUND EQU $77 Voice was not found
********************************************************************
* Constant definitions for general purpose
* Port data direction registers (offset from $1000)
REGBAS EQU $1000
DDRC EQU $07 Data direction for port C (0=Input, 1=Output)
OC1M EQU $0C OC1 Action Mask register
OC1D EQU $0D OC1 Action Data register
TIC1 EQU $10 IC1 register (16 bits)
TOC1 EQU $16 OC1 register (16 bits)
TOC3 EQU $1A OC3 register (16 bits)
TCTL1 EQU $20 OM2,OL2;OM3,OL3;OM4,OL4;OM5,OL5
TCTL2 EQU $21 -,-;EDG1B,EDG1A;EDG2B,EDG2A;EDG3B,EDG3A
TMSK1 EQU $22 OC1I,OC2I,OC3I,OC4I,OC5I,IC1I,IC2I,IC3I
TFLG1 EQU $23 OC1F,OC2F,OC3F,OC4F,OC5F,IC1F,IC2F,IC3F
*PVIC1 EQU $00E8 EVB pseudo-vector for IC1
TMRREG EQU $0E Timer counter register (16 bits)
* A/D controller and reading defines
ADCTL EQU $30
ADR1 EQU $31
ADR2 EQU $32
ADR3 EQU $33
ADR4 EQU $34
Option EQU $39
****************************************************************************
* Voice clip indices
* /* Voice response clips */
vLIKETOAST EQU %00000000 /* Would you like some toast? */
vTOAST EQU %00001100 /* toast */
vYES EQU %00010100 /* yes? */
vNO EQU %00011011 /* no */
vOKAY EQU %00100010 /* okay */
vHOWLIGHT EQU %00101001 /* how light? */
vLIGHT EQU %00110010 /* light */
vMEDIUM EQU %00111010 /* medium */
vDARK EQU %01000001 /* dark */
vUSING EQU %01001001 /* using setting *?
vEND EQU %01010011 /* end */
vLOWERING EQU %01011011 /* lowering */
vRAISING EQU %01100011 /* raising */
vDONE EQU %01101011 /* done */
vSLOWER EQU %01110010 /* slower */
vFASTER EQU %01111010 /* faster */
vWHAT EQU %10000010 /* I'm sorry, what? */
vPLSSAY EQU %10001101 /* Please say -- */
vNOBREAD EQU %10010110 /* I should toast what? */
****************************************************************************
* Speech recognition indicies
* /* Word Recognition indicies (bcd) */
wYES EQU $01 #define wYes '01'
wNO EQU $02 #define wNo '02'
wTOAST EQU $03 #define wToast '03'
wLIGHT EQU $04
wMEDIUM EQU $05
wDARK EQU $06
wEND EQU $07 #define wEnd '07'
wLASTWORD EQU $07 #define wLastWord '07'
srMAXTRAIN EQU 3 #define srMaxTrain 3
srTHRESH EQU $80 #define srThresh 0x40
*********************************************************************************
* I/O port mapping for voice chip and motion detection
VOCPORT EQU $1004 Port B
isdENBAR EQU $1003 Port C port gPlayClip[0];
isdENMASK EQU %10000000 Bit 7
isdEOM EQU $1000 Port A port gEOM[0];
isdEOMMASK EQU %00000001 Bit 0
MOTPORT EQU $1000 Port A port gMotion[0];
MOTMASK EQU %00000010 Bit 1
*********************************************************************************
* Bread Platform pulse widths and control data
OC3_OC1 EQU %00100000 OC1 controls A[5]
OC3_CTL EQU %00100000 OC3 clears on compare
PW_RAISE EQU $0D80
PW_LOWER EQU $0480
BREAD_DLY EQU 32
tLIGHT EQU $91 Treshold for light toast
tMEDIUM EQU $93 Treshold for medium toast
tDARK EQU $95 Treshold for dark toast
*********************************************************************************
* Global Variables
ORG $0048
* The only global variable. And it could even go away if I wanted to use
* input capture. But I don't so it won't. 8)
motLASTST RMB 1 Last known state of motion sensor
srINITED RMB 1 1 == INIT_SR has been called, 0 otherwise
TOASTLEVEL RMB 1 Toast level (threshold for temp IC)
***********************************************************************
* Interrupt vectors, including RESET and IC1
ORG $FFFE RESET: jump to start of program
FDB MAIN
ORG $FFEE IC1 ISR
FDB IC1ISR
ORG $F800
* If we're not on the EVB, pass all the Buffalo functions to NULLFCN.
NULLFCN RTS
* BUFFALO I/O routines
* Must be defined here because they must *follow* NULLFCN's definition
* The $FFxx addresses are the addresses of the functions when used with the EVB
OUTA EQU NULLFCN $FFB8 Output ASCII character in A
OUT1BSP EQU NULLFCN $FFBE Output 1 byte in hex, followed by a space
OUT1BYT EQU NULLFCN $FFBB Output 1 byte in hex
OUTCRLF EQU NULLFCN $FFC4 Output carriage return and line feed
OUTSTRG EQU NULLFCN $FFC7 Output string terminated by $04, incl. CR/LF
OUTSTRG0 EQU NULLFCN $FFCA Output string terminated by $0F, excl. CR/LF
***********************************************************************
* void main(void)
* {
* int w;
MAIN LDS #$0047
JSR INIT Init();
LDAA #'M
JSR OUTA
JSR OUTCRLF
* while(1) {
MAINLOOP LDAA #'L
* JSR OUTA
* JSR OUTCRLF
JSR WORDRECOG if (WordRecognized()) {
BEQ ML1
JSR GETWORD w = GetWord();
PSHA
TSX
JSR OUT1BSP
PULA
SUBA #wTOAST if (w == wToast)
BNE MAINLOOP
JSR MAKETOAST MakeToast();
LDAA #'/
JSR OUTA
LDAA #'P
JSR OUTA
BRA MAINLOOP
* }
ML1 JSR MOTDETECT else if (MotionDetected()) {
BEQ MAINLOOP
JSR RESPONDMOT RespondToMotion();
BRA MAINLOOP
* }
* }
* }
***********************************************************************
* void Init(void)
* {
INIT JSR INIT_VR InitVoiceResponse();
JSR INIT_SR InitSpeechRecognition();
JSR INIT_BREAD
CLRA
STAA motLASTST gLastMotState = 0;
RTS
* }
****************************************************************************************
* void InitSpeechRecognition(void)
* {
INIT_SR LDAB #wLASTWORD int i=wLastWord;
* int j;
CLRA /* We're not fully initialized yet */
STAA srINITED
JSR srCLEAR srClearPatterns();
* /* i == Accumulator B */
* /* j == Accumulator A */
* /* Train vocabulary */
ISR4 TBA while (i) {
BEQ ISR1
LDAA #srMAXTRAIN j = srMaxTrain;
ISR3 BEQ ISR2 while(j) {
LDX #ISR_MSG1 puts("Please train word: ");
PSHA
JSR OUTSTRG
** TBA puts(words[i]);
** ADDA #'0
** JSR OUTA
LDX #srWORDS /* X <= &words[0].s */
PSHB
DECB /* words is 0-based */
LSLB
ABX /* X <= &words[0].s + 2*i */
LDX 0,X /* X <= words[i].s */
JSR OUTSTRG
PULB
LDAA #vPLSSAY
JSR SAY
LDX #vWORDS /* X <= &words[0].v */
ABX /* X == &words[i].v */
DEX /* words is 0-based */
LDAA 0,X /* A <= words[i].v */
JSR SAY
TBA srTrainWord(i);
JSR srTRAINWORD
PULA j--;
SUBA #1
BRA ISR3 }
ISR2 SUBB #1 i--;
BRA ISR4 }
* /* Activate recognition circuit */
ISR1 LDAA #srRDSTATUS outs(srRdStatus);
JSR OUTS
ISR5 JSR INST while (inst() != srRdyCmd);
SUBA #srRDYCMD
BNE ISR5
LDAA #srWRINPUT outs(srWrInput);
JSR OUTS
LDAA #srCMDRECOG outk(srCmdRecog);
JSR OUTK
JSR INK ink();
LDAA #1 /* We're initialized now */
STAA srINITED
RTS
* }
************************************************************************************
* void InitVoiceResponse(void)
* {
INIT_VR LDX #$1000
BSET isdENBAR,X isdENMASK gISDEnableBar = 1;
BSET DDRC,X isdENMASK /* Set pin as output */
BSET isdENBAR,X isdENMASK gISDEnableBar = 1;
RTS
* }
*********************************************************************************
* bool WordRecognized(void)
* {
WORDRECOG LDAA #'W
* JSR OUTA
* JSR OUTCRLF
LDAA #srRDSTATUS outs(srRdStatus);
JSR OUTS
JSR INST st = inst();
SUBA #srRDYCMD if (st == srRdyCmd) {
BNE WR1
* JSR BEGINRECOG
LDAA #1 return 1;
RTS
* } else
WR1 CLRA return 0;
RTS
* }
***************************************************************************
BEGINRECOG LDAA #'B
* JSR OUTA
* JSR OUTCRLF
LDAA #srRDSTATUS outs(srRdStatus);
JSR OUTS
BR1 JSR INST while (inst() != srRdyCmd);
SUBA #srRDYCMD
BNE BR1
LDAA #srWRINPUT outs(srWrInput);
JSR OUTS
LDAA #srCMDRECOG outk(srCmdRecog);
JSR OUTK
JSR INK ink();
LDAA #20
BR2 NOP
NOP
SUBA #1
BNE BR2
LDAA #srRDSTATUS outs(srRdStatus);
JSR OUTS
JSR INST
RTS
*************************************************************************
* bcd GetWord(void)
* {
* bcd w;
GETWORD LDAA #'G
JSR OUTA
* JSR OUTCRLF
LDAA #srRDSTATUS outs(srRdStatus);
JSR OUTS
GW1 JSR INST while(inst() != srRdyCmd);
SUBA #srRDYCMD
BNE GW1
LDAA #srWRINPUT outs(srWrInput);
JSR OUTS
LDAA #srCMDRESULT outk(srCmdResult);
JSR OUTK
JSR INK ink();
* /* Read which word was read */
JSR READLOHI ReadLoHi(&w);
PSHA
TSX
JSR OUT1BSP
* /* Read score (required) and throw away */
JSR READLOHI ReadLoHi(NULL);
PSHA
TSX
JSR OUT1BSP
JSR BEGINRECOG
PULB /* B <= score */
PULA /* A <= word */
SUBA #srTOOFAST
BNE GW2
LDAA #vSLOWER
JSR SAY
BRA GETWORD
GW2 ADDA #srTOOFAST
SUBA #srTOOSLOW
BNE GW3
LDAA #vFASTER
JSR SAY
BRA GETWORD
GW3 ADDA #srTOOSLOW
SUBA #srNOTFOUND
BNE GW4
LDAA #vWHAT
JSR SAY
BRA GETWORD
GW4 ADDA #srNOTFOUND /* A is word (again) */
PSHA
TBA
SUBA #srTHRESH
BLO GW5
PULA
LDAA #vWHAT
JSR SAY
BRA GETWORD
GW5 LDAA #'/
JSR OUTA
LDAA #'G
JSR OUTA
JSR OUTCRLF
PULA
RTS return w;
* }
*****************************************************************************
* void ReadLoHi(bcd *w)
* {
READLOHI LDAA #'R
JSR OUTA
LDAA #'L
JSR OUTA
LDAA #'H
JSR OUTA
JSR OUTCRLF
LDAA #srRDSTATUS outs(srRdStatus);
JSR OUTS
RLH1 JSR INST while (inst() != srRdyLo);
SUBA #srRDYLO
BNE RLH1
LDAA #srRDOUTPUT outs(srRdOutput);
JSR OUTS
JSR INK w->lo = ink();
PSHA
LDAA #%101 outs(101);
JSR OUTS
LDAA #%001 outs(001);
JSR OUTS
LDAA #%000 outs(000);
JSR OUTS
LDAA #srRDSTATUS outs(srRdStatus);
JSR OUTS
RLH2 JSR INST while(inst() != srRdyHi);
SUBA #srRDYHI
BNE RLH2
LDAA #srRDOUTPUT outs(srRdOutput);
JSR OUTS
JSR INK w->hi = ink();
LSLA
LSLA
LSLA
LSLA
PULB
ABA
PSHA
LDAA #%101 outs(101);
JSR OUTS
LDAA #%001 outs(001);
JSR OUTS
LDAA #%000 outs(000);
JSR OUTS
LDAA #srRDSTATUS outs(srRdStatus);
JSR OUTS
JSR INST inst();
PULA
RTS
* }
***********************************************************************************
* void srTrainWord(bcd index)
* {
srTRAINWORD PSHA /* Save index from register A */
LDAA #'T
JSR OUTA
LDAA #'W
JSR OUTA
TSX
JSR OUT1BSP
JSR OUTCRLF
LDAA #srRDSTATUS outs(ReadStatus);
JSR OUTS
TW_1 JSR INST while(inst() != ReadyForCommand);
SUBA #srRDYCMD
BNE TW_1
LDAA #srWRINPUT outs(WriteInput);
JSR OUTS
LDAA #srCMDTRAIN outk(CmdTrain);
JSR OUTK
JSR INK
LDAA #srRDSTATUS outs(ReadStatus);
JSR OUTS
TW_2 JSR INST while(inst() != ReadyForLo);
PSHA
SUBA #srRDYCMD
BEQ TW_CONFZD
PULA
SUBA #srRDYLO
BNE TW_2
LDAA #srWRINPUT outs(WriteInput);
JSR OUTS
PULA
PSHA
ANDA #$0F outk(loDigit);
JSR OUTK
JSR INK
LDAA #srRDSTATUS outs(ReadStatus);
JSR OUTS
TW_3 JSR INST while(inst() != ReadyForHi);
SUBA #srRDYHI
BNE TW_3
LDAA #srWRINPUT outs(WriteInput);
JSR OUTS
PULA
LSRA
LSRA
LSRA
LSRA
ANDA #$0F outk(hiDigit);
JSR OUTK
JSR INK
LDAA #srRDSTATUS outs(ReadStatus);
JSR OUTS
TW_4 JSR INST inst();
SUBA #srRDYCMD while (inst() != ReadyCommand); /* Wait for word */
BNE TW_4
RTS
TW_CONFZD PULA /* Restore the stack! */
LDX #CONFZD_MSG printf("Voice chip confused... trying again.\n");
JSR OUTSTRG
PULA
JMP srTRAINWORD
* }
********************************************************************************
* void srClearPatterns(void)
* {
srCLEAR LDAA #srRDSTATUS outs(ReadStatus);
JSR OUTS
CLR_1 JSR INST while(inst() != ReadyForCommand);
SUBA #srRDYCMD
BNE CLR_1
LDAA #srWRINPUT
JSR OUTS
LDAA #srCMDRESET outk(CmdReset);
JSR OUTK
JSR INK ink();
LDAA #2000
CLR_2 NOP
NOP
SUBA #1
BNE CLR_2
LDAA #srRDSTATUS outs(ReadStatus);
JSR OUTS
JSR INST
RTS
* }
****************************************************************************
* void MakeToast(void)
* {
* /* Check preconditions, like bread in toaster */
MAKETOAST LDAA #'M
JSR OUTA
LDAA #'T
JSR OUTA
JSR OUTCRLF
JSR CHECKBREAD
BNE MTOK
LDAA #vNOBREAD
JSR SAY
RTS
* /* Ask what toasting level */
MTOK LDAA #vHOWLIGHT
JSR SAY
MT0_1 JSR WORDRECOG
BNE MT0_1
JSR GETWORD /* Light toast? */
SUBA #wLIGHT
BNE MT0_2
LDAA #tLIGHT
STAA TOASTLEVEL
LDAB #vLIGHT
BRA MTCONT
MT0_2 ADDA #wLIGHT /* Dark toast? */
SUBA #wDARK
BNE MT0_3
LDAA #tDARK
STAA TOASTLEVEL
LDAB #vDARK
BRA MTCONT
MT0_3 LDAA #tMEDIUM /* Medium toast (default) */
STAA TOASTLEVEL
LDAB #vMEDIUM
MTCONT PSHB
LDAA #vUSING
JSR SAY
PULB
TBA
JSR SAY
LDAA #vLOWERING Say(vLowering);
JSR SAY
JSR LOWER
* while(1) {
MT3 JSR CHECKTEMP if (TempThreshReached())
BNE MT4 break;
JSR WORDRECOG if (!WordRecognized())
BEQ MT3 continue;
JSR GETWORD if (GetWord() != wEnd)
SUBA #wEND
BNE MT3 continue;
BRA MT4 break;
* }
MT4 LDAA #vRAISING Say(vRaising);
JSR SAY
JSR RAISE
LDAA #vDONE Say(vDone);
JSR SAY
RTS
* }
****************************************************************************
* ***** CHRIS Read bread sensor here and put the number of slices loaded
* ***** into accumulator A
* For now, just lie and say that the toaster if fully loaded
CHECKBREAD NOP
* Set the ADCTL to read PE1 (left eye)
LDX #REGBAS
*** Eye 1 seems to be misbehaving. It is being software disabled for now.
*** Misbehaving manifests itself by causing the microC to start executing
*** instructions from anywhere within its memory, not nec. in PC order.
**Eye1 LDAA #%10000001
** STAA ADCTL,X
** JSR GetVal
* ADR1 is one of 4 AD values we can use
** LDAA ADR1,X
* I hope this constant works!!!
** SUBA #$80
** BLS BreadIn1
** CLRA
** BRA Eye2
**BreadIn1 LDAA #1
** PSHA
Eye2 LDAA #%10000010
STAA ADCTL,X
JSR GetVal
* ADR1 is one of 4 AD values we can use
LDAA ADR1,X
* I hope this constant works!!!
SUBA #$80
BLS BreadIn2
CLRA
** PULA
RTS
BreadIn2 NOP
** PULA
LDAA #1
RTS
****************************************************************************
* ***** CHRIS Read the temperature IC. If the temperature is
* above some threshold, return 1 in accumulator A. Else, return
* 0 in A. For now, we'll lie and say that the temperature
* has not been reached yet.
CHECKTEMP NOP
LDX #REGBAS
* If counter is less than 25K, there's probably IC1 noise on the line. Don't
* try to read the A/D port
LDD TMRREG,X
SUBD #25600
BLO CHKTEMP_END
* Set the ADCTL to read PE7
LDX #REGBAS
LDAA #%10000111
STAA ADCTL,X
JSR GetVal
* ADR1 is one of 4 AD values we can use
LDAA ADR1,X
STAA VOCPORT For debugging, output the value to the ISD's bus (port B)
SUBA TOASTLEVEL
BHS Hot
CLRA
RTS
Hot LDAA #1
RTS
CHKTEMP_END CLRA
RTS
***********************************************************
* Common ADC routine to wait until ADC is ready with value.
GetVal NOP
LDX #REGBAS
WaitAD LDAA Option,X
ANDA #%10000000
BEQ WaitAD
ReadAD LDAA ADCTL,X
ANDA #%10000000
BEQ ReadAD
RTS
****************************************************************************
* void Say(int vIndex)
* {
SAY STAA VOCPORT outva(vIndex); /* Output voice clip address */
LDAA #'S
JSR OUTA
JSR OUTCRLF
LDX #$1000 gPlayClip = 0;
BCLR isdENBAR,X isdENMASK
LDAA #20 /* The low pulse must be long enough for the chip to catch it */
S0 NOP
NOP
SUBA #1
BNE S0
BSET isdENBAR,X isdENMASK gPlayClip = 1;
BRSET isdEOM,X isdEOMMASK * while (!gEOM);
BRCLR isdEOM,X isdEOMMASK * while (gEOM); /* low for 16 ms */
* /* If the toaster's voice triggered
* the speech chip, reset the recognition */
LDAA #srRDSTATUS outs(srRdStatus);
JSR OUTS
JSR INST st = inst();
SUBA #srRDYCMD if (st != srRdyCmd)
BNE S1
LDAA srINITED if (srInited)
BEQ S1
JSR BEGINRECOG
S1 RTS
* }
*********************************************************************************
* bool MotionDetected(void)
* {
MOTDETECT LDAA #'O
* JSR OUTA
* JSR OUTCRLF
LDAA MOTPORT
ANDA #MOTMASK
BEQ MD1 if (gMotion) {
LDAA motLASTST if (!gLastMotState) {
BNE MD2
LDAA #1 gLastMotState = 1;
STAA motLASTST
RTS return 1;
* }
MD1 LDAA motLASTST } else if (gLastMotState) {
BEQ MD2
CLRA
STAA motLASTST gLastMotState = 0;
* }
* }
MD2 CLRA
RTS return 0;
* }
**********************************************************************************
* void RespondToMotion(void)
* {
RESPONDMOT LDAA #'R
JSR OUTA
LDAA #'M
JSR OUTA
JSR OUTCRLF
LDAA #vLIKETOAST Say(vLikeToast);
JSR SAY
RM1 JSR WORDRECOG
BNE RM1
JSR GETWORD /* Yes */
SUBA #wYES
BNE RM2
JSR MAKETOAST
RTS
RM2 LDAA #vOKAY
JSR SAY
RTS
* }
*********************************************************************************
INIT_BREAD NOP
* Set up IC1 to override microcontroller's grip on the servo
* and raise the bread platform
* These lines are needed to set up the IC1's ISR on the EVB
* LDAA #$7E Jump (extended) opcode
* STAA PVIC1
* LDX #IC1ISR
* STX PVIC1+1
LDX #REGBAS
* Set up OC3 (A[5]) to drop pulse at successful output compare
BCLR TMSK1,X %11111000
BCLR TFLG1,X %11111000 Writing a 1 clears the bits
LDAA #OC3_CTL
STAA TCTL1,X
LDD #PW_RAISE
STD TOC3,X
* Set up OC1 to raise pulse at timer overflow
LDAA #OC3_OC1 OC1 affects OC3
STAA OC1M,X
STAA OC1D,X OC1 writes a 1 to OC3
CLRA
CLRB
STD TOC1,X Successful compare at CLK == 0
BSET TMSK1,X %00000100 IC1 ISR
BSET TFLG1,X %00000100 Clear IC1 flag (if any)
LDAA #%00110000 Capture on any edge
STAA TCTL2,X
* Here's where we initialize the ADC.
LDAA #%10000000
STAA Option,X
CLI Enable interrupts
** JSR RAISE
RTS
*********************************************************************************
RAISE LDX #REGBAS
** LDAA #OC3_CTL Enable OC3
** STAA TCTL1,X
** LDAA #OC3_OC1 Enable OC1
** STAA OC1M,X
LDD #PW_RAISE Raise bread platform
STD TOC3+REGBAS
** JSR PAUSEBREAD
** CLRA
** STAA TCTL1,X Disable OC3
** STAA OC1M,X Disable OC1
RTS
*********************************************************************************
LOWER LDX #REGBAS
** LDAA #OC3_CTL Enable OC3
** STAA TCTL1,X
** LDAA #OC3_OC1 Enable OC1
** STAA OC1M,X
LDD #PW_LOWER
STD TOC3+REGBAS
** JSR PAUSEBREAD
** CLRA
** STAA TCTL1,X Disable OC3
** STAA OC1M,X Disable OC1
RTS
*********************************************************************************
IC1ISR LDX #REGBAS
LDD #PW_RAISE
STD TOC3,X
BSET TFLG1,X %00000100 Clear IC1 flag
RTI
*********************************************************************************
PAUSEBREAD LDAA #BREAD_DLY
PB1 PSHA
JSR PAUSE32
PULA
DECA
BNE PB1
RTS
****************************************************************************
INK LDX #$1000 Set direction bits to input
BCLR DDRC,X #KMASK
LDAA KBUS Read data from K bus
ANDA #KMASK Discard non-K bus data
RTS
****************************************************************************
INST JSR INK Status register is on K bus
ANDA #%00000011 Status bits are lower two
RTS
****************************************************************************
OUTK LDX #$1000
BSET DDRC,X #KMASK
PSHA Save data to be written
TSX
LDAA KBUS Read current value
ANDA #KMASK^$FF Save non-K bus part
ORA 0,X Add K-bus part
STAA KBUS Write data
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
PULA Clean up stack
RTS
****************************************************************************
OUTS LDX #$1000
BSET DDRC,X #SMASK
LSLA 00000SSS -> 0000SSS0
LSLA 0000SSS0 -> 000SSS00
LSLA 000SSS00 -> 00SSS000
LSLA 00SSS000 -> 0SSS0000
PSHA Save data to be written
TSX
LDAA SBUS Read current value
ANDA #SMASK^$FF Save non-S bus part
ORA 0,X Add S-bus part
STAA SBUS Write data
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
NOP
PULA Clean up stack
RTS
PAUSE NOP
NOP
SUBA #1
BNE PAUSE
RTS
PAUSE32 LDAA #$FFFF
P32 DECA
BNE P32
RTS
vWORDS FCB vYES,vNO,vTOAST,vLIGHT,vMEDIUM,vDARK,vEND
srWORDS FDB srStrYES,srStrNO,srStrTOAST,srStrLIGHT,srStrMEDIUM,srStrDARK,srStrEND static char* words[] = {
srStrYES FCC 'yes' "yes",
FCB $04
srStrNO FCC 'no' "no",
FCB $04
srStrTOAST FCC 'toast' "toast",
FCB $04
srStrLIGHT FCC 'light' "light",
FCB $04
srStrMEDIUM FCC 'medium' "medium",
FCB $04
srStrDARK FCC 'dark' "dark",
FCB $04
srStrEND FCC 'end' "end"
FCB $04
* };
ISR_MSG1 FCC 'Train word: '
FCB $04
CONFZD_MSG FCC 'Voice chip confused... returning to command loop.'
FCB $04
Once the green indicator lights and the toaster has said the word 'end', speak the word 'end' clearly to the toaster. Say the word exactly as you will say it later when you will ask for toast. In response, the toaster will ask you to say a word again, either the same word or another word. Each word will be said three times, and there are seven words. Once you have said the last word three times (the word 'yes'), the toaster enters its main loop.
To make toast
The toaster waits for either the user to say a word or for someone to walk
by. If someone walks by, the toaster asks that person, "Would you like
some toast?". If the person responds with 'no', the toaster will answer
"Okay" and re-enter the main loop. Else, if the person responds with
'yes', the toaster will begin its toast making process. This is the same
process as can be entered if the user asked the toaster for 'toast' from
the main loop.
At the start of the toast-making process, the toaster checks to see if there is bread in the toaster. If there is no bread, the toaster asks, "What should I toast?" and returns to the main command loop. Else, the toaster asks, "How light?", and expects a response of either 'light', 'medium', or 'dark'. If the toaster cannot understand what is said, it will take 'medium' as the default.
After a toast level has been set, the toaster will lower the bread platform and begin toasting. While toasting, the user may abort the process by saying 'end'. If the user does not abort, the toaster will raise the bread platform when the desired temperature threshold is met. After raising the bread platform, the toaster announces that it is done, and re-enters the command loop.
One note about using this toaster: after making one slice of toast, the toaster becomes very hot. Subsequent slices of toast will not be prepared fully as the first slice because the temperature sensor watches for a threshold, as opposed to differential temperature change. Thus, it is usually a good idea to let the toaster cool off for about five minutes between making batches of toast.
Corey's tasks
All of the VIS, including determining how the HM2007L speech recognition
system worked. Versions 0.1 through 1.6e2 of TOS. Wirewrapping of the
Main Controller Logic Board. Motion Sensor. Minor assembly of external
logic box.
Chris' tasks
Bread and temperature sensors. Bread Platform motor. Procurement of and
modifications to the toaster. TOS v1.4 and up. Major assembly of
external logic box. Soldering of the SDS Controller Board.
1 The "logic box" is a metal box with the same
footprint as the toaster. All circuitry is housed in this box to protect
the circuitry from the extreme temperatures of the toaster's heating
coils.
2 Of course, there is always the option of ordering food
to be delivered. But there still is the same fundamental problem: how to
do this without activating any more buttons or watching any more
lights.
3 There are two modes of operation for the HM2007L: CPU
mode and manual mode. In CPU mode, the chip obeys commands given on the
I/O bus. In manual mode, a keypad is connected to the I/O bus, and the
chip scans the keypad directly, obeying commands pressed by the user. The
distributor has constructed a test bed for the HM2007L in manual mode, but
has not done the same for CPU mode yet.
4 One degree Kelvin is equal to one degree Celcius. Zero
K = -273 deg C.
5 We had originally mounted two photoresistors, but due
to some bizarre A/D port trouble on the microcontroller, we were forced to
deactivate one of the photoresistors.
Corin Anderson |
corin@the4cs.com
Last modified: September 8, 1996