On first glance, the toaster looks like a toaster of the classic 1950s era, complete with polished aluminum sides. Inside, however, is a completely different story. In an external logic box 1 is a microcontroller that processes the sensor input and generates appropriate responses.
Sensors include a motion sensor to detect a toast-hungry person; photoresistors to detect the presence of bread to be toasted; and a temperature sensor to measure the energy transmitted to the bread while toasting. For the user interface, there is speech recognition and voice response. The speech recognition allows the user to simply speak commands to the toaster. The voice response module provides friendly feedback to the user based upon the user's selection.
To ensure the safety of the toast connoisseur, the power lines in the toaster are properly insulated. Additionally, to ensure the safety of the toaster, the microcontroller will be shielded from the line current switch by external circuitry.
The solution we propose is to use a voice activated user interface, where the food preparing device is controlled by commands spoken by the Computer Scientist. In turn, the food preparing device will also respond to the Computer Scientist with prerecorded voice clips. The typical Computer Scientist conducts most of his or her correspondence via electronic mail, so the vocal cords are hardly stressed. The voice user interface is the best.
As a prototype for this new user interface, we have designed and built one of the simpler food preparing devices under the new paradigm: we have constructed a Talking Toaster.
Another big win from modularity is the debugging of the final assembled system. Given that each component had been tested and debugged on its own, if the final assembly did not work, we could concentrate our debugging effort on the interconnection of each component. Further, we could add components one by one to the final assembly, reducing the number of interconnections that needed to be checked should something go wrong.
While designing and testing these systems, we noticed that the voice clips played back sounded noisy. Through some experiments with an oscilloscope, we determined that both the power supply and evaluation board (which housed the microcontroller during initial prototyping) were electronically noisy. To deaden this noise, bypass filter capacitors (1000mF) were placed from power to ground near the voice response and speech recognition chips.
Voice Response
The voice response subsystem was implemented with an Information Storage
Devices' ISD1000A analog memory chip. This chip allowed for up to 20
seconds of response to be stored for playback. Further, the storage was
nonvolatile, so no battery backup was necessary.
There were two aspects of the voice response system. The first was playing responses back at the appropriate times. The second was recording the responses to be used in the toaster. Because the responses are not to be changed by the user, there is no recording capability in the toaster's circuit. This greatly simplifies the circuit, mainly because the play/record pin on the voice chip may be permanently tied to play. The other alternative, where the microcontroller sets and clears the play/record pin, is troublesome during debugging because the microcontroller may hang and accidentally assert the record signal and erase the voice chip.
The play-only circuit using the ISD1000A chip may be found in the upper right corner of the Main Controller Logic Board schematic in Appendix A. To specify which voice clip to play, the microcontroller sends an 8-bit address out on its port B. To play the voice clip, the microcontroller pulses the ISD1000A's chip enable (CE) signal using port C[7]. To signal that the end of the message has been reached, the ISD1000A pulses the end of message pin (EOM) for 16 milliseconds.
The recording circuit is not pictured, but is not any more complex than what is shown. The only changes are the addition of an electret microphone and some passive components on the MIC line, and the microcontroller driving the play/record (P/R) pin.
To experiment with the voice response system, an assembly code module was written that interfaces the microcontroller to the ISD1000A. Later in the construction of the toaster, this module became quite valuable as an instrument with which to record the voice responses. To aid in debugging, the ISD1000A driver module was written using the same microcontroller ports as would be used in the final design. That is, no wires had to be moved, etc., when first debugging the entire design, and then verifying just the voice response system.
Speech Recognition
The speech recognition subsystem was built using Hualon Microelectronics
Corporation's HM2007L speech recognition chip. This chip allowed for
words to be recognized from a vocabulary of up to 40 1-second long words.
The vocabulary was stored on an external 8K SRAM. This SRAM was not
powered by a battery, so the vocabulary was lost when the toaster was
unplugged. While this may not be ideal for the Computer Scientist, this
removed a design constraint that otherwise would have hampered the
project.
To use the speech recognition system of the HM2007L, the user must train their voice prints on the chip. In the current version of TOS, the user is instructed on how to do this when the toaster is first plugged in. For each word that is to be recognized, the microcontroller asks the user to speak that word. Because the user may say the word differently (i.e., with slightly different inflections, etc.), the user is asked to say the word more than once (usually three times). For each time the user says the word, the HM2007L integrates this word into a neural network (this network is stored in the off-chip SRAM). Later, in recognition mode, the HM2007L tries to match the spoken word against other words in its neural net. If a match is made, the index of that word in the vocabulary is returned. If no match is found, or if the user spoke too quickly or too slowly, an appropriate error code is returned.
Thus, the HM2007L does not recognize a spoken word as an actual word, but rather as sounding like a word that it knows about. The HM2007L has no a priori knowledge of what the word 'toast' should sound like.
The implementation of the speech recognition system was by far the most difficult engineering feat in the entire project. The chip came with a data sheet, but the information contained therein was in many instances unclear and even incorrect. The distributor of the chips had no experience with using the chips in CPU mode 3, so we were on our own. Fortunately, after two weeks of intensive experimenting and designing, we had an understanding of the basic program flow required to train and recognize words.
The HM2007L is a 52-pin PLCC chip, with a 48-pin DIP counterpart, the HM2007P, and a bare-pad version, simply HM2007. The HM2007L has 4 I/O ports, a microphone system, and several control pins. To communicate with the 8K SRAM, there is a 13-bit address bus and a 8-bit data bus (two of the four I/O ports), as well as a memory read/write pin and a memory enable pin. To communicate with the microcontroller, there is a 4-bit wide K-bus, used for passing data to and from the HM2007L, and a 3-bit S-bus, used for sending commands to the HM2007L (mainly commands to control the direction and meaning of the K-bus). Because we had only two of these chips, and no more were expected to be available from the manufacturer until after the end of the quarter, we took great care in protecting our chips from possible shorts. The S-bus and K-bus were connected to the microcontroller's C port via 10kOhm resistors. If both chips tried to drive the lines at different logic values, at least there would be no direct short through the chips.
Like the ISD1000A, a separate assembly code module was written to experiment and debug the speech recognition chip. And like its counterpart with the ISD1000A, this module was invaluable in debugging the full prototype.
The next issue was modularity. From the start, easy access to the toaster's working parts had to be engineered into the design to facilitate easy debugging and experimentation. Different positions for the temperature and bread sensors were tried to seek the optimal position. The motor used to raise and lower the bread platform needed to sit in exactly the right position.
To aid in this modularity issue, a quick-release toaster cover was fabricated. This involved using some attachment screws on the toaster handles and removing excess metal from the chrome cover. The feet of the toaster were fixed to the toaster directly, as opposed to being fixed to the chrome cover. The toaster, in turn, was secured to the external logic box by four thumb screws. In total, these modifications allowed us to debug the entire design both with and without the toaster, depending upon what was easier at the time.
Temperature Sensor
The first sensor installed was the temperature
sensor. The temperature sensor sampled the current temperature inside the
toaster, which was read through the A/D port on the microcontroller. When
a threshold temperature was reached (determined by the toast setting the
user had indicated: light, medium, or dark), the microcontroller would
raise the bread platform.
The temperature sensing device we used was Analog Device's AD590J, a temperature transducer. The particular part we used was an integrated circuit mounted in a metal can (AD590JH). The AD590J reported the temperature by outputting a current in mA equal to the temperature in degrees Kelvin 4. Thus, at room temperature (298 K), the AD590J would output 298mA of current.
The operating range of the AD590J is between 218K and 423K (-55 deg and 150 deg C). While this is a wide range extending into high temperatures, the interior of the toaster raises far hotter than this. Through experimenting, we found the best position for the sensor, one where it did not reach its upper limit before the toast was done, was just beneath the toaster.
Once the temperature sensor was mounted, and toasting tests were begun, we began to notice erratic behavior from the AD590J. As it turned out, the A/D port on the microcontroller was experiencing terrible noise every 32ms, just when the servo motor's controlling signal was reset. The noise manifested itself as ringing in the lower two or three bits of the A/D channel. While this should not have much of an impact, it did in this case, because we had lower resolution on our temperature sensor (see below). Thus, we needed a fix. The solution: simply ignore temperature readings during this noisy period. Because the noise lasted for less than 12ms at every timer rollover, we could simply look at the microcontroller's timer and ignore the data if the counter were less than 25k.
Bread Sensor
In order to make toast, the toaster must assure that there is bread
loaded. To check the bread loaded condition, we mounted photoresistors 5 on the external logic box, looking up through the
bread slots in the toaster. Under normal conditions, light in the room
would filter through the toaster and strike the photoresistors, keeping
their resistance at some nominal level. When bread is inserted, the light
no longer makes it all the way down to the photoresistors, and their
resistance changes. Using the circuit found in Appendix A (SDS Controller board), the change in
resistance was converted into a change of voltage, which was in turn read
by the microcontroller's A/D port.
Motion Sensor
The motion sensor turned out to be easier than was
originally planned. In the final version of the toaster, a motion sensor
from a security porch light was used. The motion sensor was a remote unit
that ran on two 9v batteries. When the unit detected motion, it would
send a radio signal to a receiver in the toaster, which would close a
relay, raising a signal on the microcontroller. The microcontroller would
respond accordingly, and set a flag. The problem is that, once motion has
been detected, the relay remains closed until the motion stops, and then
for 12 seconds after that. During that time, the toaster should not be
assuming that it needs to react to the motion. Thus, a flag is set and
not cleared until the relay has been opened again.
Bread Platform
In parallel with the development of the
sensors was the work on the bread platform. After many false starts,
including a long and windy path of learning how stepper motors work, we
made the observation that a servo motor would do exactly the right job for
us. Where a stepper (or other non-servo motor) would easily be driven by
the microcontroller, there would be a need for feedback in the toaster's
up and down positions. That is, when the bread platform reached its
maximum or minimum height, the microcontroller would have to be
interrupted. This would involve another two or three I/O pins that we
could not afford. On the other hand, using a servo would eliminate the
need for this feedback. By simply sending an accurate pulse-width
modulated (PWM) signal to the motor, the shaft would turn to exactly the
right position, and hold there as long as the PWM signal was held.
SDS Controller Board
All the sensors mounted in and on the external logic box needed some way
to be powered and/or interfaced into the microcontroller. Because the
microcontroller and other ICs were wirewrapped, a separate board for the
SDS seemed most logical. Thus, we have the SDS Controller Board. A
schematic for the SDS Controller Board appears in Appendix A.
Once again recalling that modularity was a key design factor, we incorporated that into the SDS Controller Board. To interface to the Main Controller Logic Board, the SDS had an 8-conductor cable terminated with an RJ-45 connector. The Main Controller Logic Board had an RJ-45 socket wirewrapped accordingly, and the two could easily be connected and disconnected as need be.
Besides offering a convenient method of connecting the sensors to the wirewrap board, the SDS Controller Board also generated the high and low reference voltages for the microcontroller's A/D converter. The A/D converter required at least 2.5v between the low and high references, so we built a voltage divider that offered 2.6v difference. This was much more than we would have liked; the AD590J had a voltage swing of 1 volt. To make the case worse, we were interested in only a fraction of the temperature sensor's range. The difference between when light toast is done and when dark toast is done is only a few degrees. Unfortunately, due to time constraints, we could not apply any remedy for this situation, and had to suffice with a few bits of resolution on the temperature sensor.
The basic flow of logic in TOS is as follows:
Initialize
Has the user said 'toast'?
Has there been movement?
If the user asked for toast, or if there has been movement, check to see
if there is bread in the toaster. If there is no bread, ask the user to
load the toaster and try again. Else, ask the user how light (light,
medium, or dark) they would like their toast. Toast the bread, and hail
the user when its all done. Return to start.
For each subsystem (speech recognition, voice response, motor control, sensors), a separate driver module was written that tested that component. As the components were completed, their drivers were added to TOS. This allowed for TOS to grow in parallel with the design work on the rest of the toaster. This parallelism was a big win for us, as it kept both of us busy, not waiting for one another.
This prototyping method also lead to the use of the EVB's Buffalo functions for displaying information on the terminal, and for setting up interrupt service routines. When the TOS was converted to be run on the wirewrapped Main Controller Logic Board, with its embedded MC68HC11E2 microcontroller, the code had to be modified (this lead to the e2 extension for versions of TOS that are wirewrap-board specific). In particular, the functions provided by buffalo were no longer available, and had to be "stubbed out". This is what we see by the NULLFCN in TOS v1.6e2. Also, the origin of the code, data, and stack were different on the 'E2 chip. Code started at address $F800. The stack had to be explicitly loaded, so the address $0047 (inside the 256 bytes of on-chip RAM) was used. Stack grows down, so there are 71 bytes of stack space available in this configuration. For global data, the origin was selected as $0048, just above the stack.
One more change necessary when porting TOS to the 'E2 chip was loading in the interrupt service routines. This is taken care of in a strange manner on the evaluation board, through the pseudo-vectors. On the 'E2 chip, we had to explicitly program the reset and IC1 ISRs in the assembly source code.
We also learned that too much planning can be as bad as too little planning. From the start, we had tried to specify certain components, like what type of motor to use for the bread platform. By attempting to nail these components down early on, we locked ourselves into a mind set that other parts would not due. This was particularly true for the bread platform motor. We had convinced ourselves that a stepper motor with a worm gear was the way to go. In reality, this method was fundamentally flawed, and a servo motor would be by far a wiser choice. We spent at least a week and a half chasing down this blind alley, mainly because we took that long to realize that we had other options.
Another lesson that we learned has already been mentioned: planning for debugging by modularity. From the very start we had several smaller components in our design, and we could set our milestones based on the progress we made with each component. We could design, build, and debug these components each on our own, without needing to find a common time that we could share in the lab. And when it came time to mesh everything together, we had few variables to check. Each component was already known to be good, so all that's left is the few wires between the modules.
There were, of course, several little changes that we could have made, and we would have benefited from the foresight had it been available. Listed with respect to the module affected, they appear below.
Temperature Sensor
We had known all along that measuring the temperature and waiting for it
to raise above a preset threshold was not the optimal solution. However,
it gave us the easiest driver code: a simple polling loop watching the A/D
port.
Given more time, we would have liked to re-implement the sensor to measure the differential temperature from when the toast making process begins. That is, after one slice of toast has been produced, and the toaster is still hot, the toaster would toast until the temperature raised by so many degrees, not simply until it reached an absolute temperature. Another method that we could have tried is to measure heat energy transmitted (the product of temperature and time).
Bread Platform
As we have lamented before, we spent a great deal of time investigating
how a stepper motor works, when the right motor for the job was sitting
idly in someone's cupboard. If we had realized much sooner that a servo
was just what we were looking for, then maybe we could have avoided our
last hour work marathon.
SDS
Much of the time in our last-hour work marathon was spent debugging the
temperature sensor's driver software and support circuit. As things turn
out, we were using A/D channel E[0] for the temperature sensor. However,
E[0] has a 10kW resistance to it, while the other A/D ports have infinite
resistance. This impacted our temperature sensor by drawing all the
current being provided, as opposed to quietly measuring the voltage
produced across the resistor in the circuit. We really wish we had known
that E[0] was a bad idea from the start.
Speech Recognition
We're really glad that we opted for this user interface, as opposed to
push buttons or even a simple clapper-type interface. It was a major
challenge to determine how the speech recognition chip worked, but it was
well worth it. We learned a lot about safe and unsafe experimental
procedures, and how to test a chip for which there is exactly one backup.
It also gave us a great deal of practice in writing assembly code for the
microcontroller, a skill that will be valuable in the future.
Main Controller Logic Board
SDS Controller Logic Board
************************************************* * * Toaster Operating System * Version 1.6e2 * * (c) 1996 Corin Anderson and Chris Setter * * CSE 477, Spring Quarter, 1996 * University of Washington * ************************************************* ******************************************************************** * Revision History * * 1.6e2 Fix applied to temperature sensor. At timer overflows, * there is noise on the A/D ports, probably caused by the PWM * signal that's driving the toaster's servo. This noise * manifests itself as a high frequency ring in the last two * bits of the temperature data. This is a major problem * the difference between light and dark toast settings are * less than 3 bits apart. The solution applied here is to * simply ignore the temperature sensor's reading during this * ringing period. The ringing lasts no longer than about * 12.5 ms, or about 25K e-cycles. This solution solves the * problem. * * 1.5e2 Modified to be used without the evaluation board. * Modifications include: * ** Globals are now at $0047 * ** Code is now at $F800 * ** RESET and IC1 interrupts are set with FDB * statement * ** Buffalo functions are replaced with NULLFCN * * 1.5 User Interface modified. No more command stuff. User * simply says "toast" and off things go. Toaster has longer * responses. * * Sensor code still being modified and tested. * * New voice chip vocabulary * * 1.4 Code added to read and respond accordingly to the bread * and temperature sensors. * * Also added code to prompt the user for toasting level. * * Never functional. Version bumped for new UI. * * 1.3 Reworked speech recognition code so the toaster won't try * to recognize its own voice (from the Say() routine). * * Added code to audibly prompt the user to train words. Also * added code to display the word on the terminal. Both of * these functions use arrays of the voice clips and words. * * New voice chip vocabulary * * 1.2 Interfaced TOS to Bread Platform servo motor. The servo * requires a pulse width modulated (PWM) control signal to * position the shaft angle. Thus, the output capture facility * is used. * * For the safety of the toaster and of the user, there is a * servo override button on the toaster. When this button is * pushed, the servo is swung up to its raise position. This * still doesn't unlock the bread platform from the motor, but * at least this will open the contacts to the heating coils * and free the user's bread. * * 1.1 Added code to disregard speech matches that have too high * of a score. With luck, this will eliminate false positives. * Unfortunately, this will also lead to false negatives. But * that's usually easier to cope with. * * 1.0 First release of functional TOS. Includes code for speech * recognition, voice response, and motion detection. Speech * recognition is limited to about 5 words trained at startup. * Voice responses are tailored for the specific command: make * toast. * * 0.1 First incarnation of TOS. Does not function properly. Has * incorrect handling of voice response chip (ISD1000A). Needs * substantial work to be functional. * ********************************************************************* ********************************************************************* * Constant definitions for Speech Recognition chip * Bus connections for Speech Recognition chip SBUS EQU $1003 S bus is C[6:4] KBUS EQU $1003 K bus is C[3:0] * Bus bit mask SMASK EQU %01110000 Middle 3 bits KMASK EQU %00001111 Lower 4 bits * States for speech recognition chip srNULL EQU %00000010 Null state srRDSTATUS EQU %00000010 Read status from K bus srRDOUTPUT EQU %00000011 Read output buffer from K bus srWRINPUT EQU %00000100 Write intput buffer on K bus * Status codes srRDYHI EQU %00000000 Ready for second (hi) nibble srRDYVOICE EQU %00000001 Ready for voice input srRDYCMD EQU %00000010 Ready to receive command srRDYLO EQU %00000011 Ready for first (lo) nibble * Command codes srCMDRECOG EQU %00000001 Recognize word srCMDTRAIN EQU %00000010 Train a pattern srCMDRESULT EQU %00000100 Read result and score srCMDRESET EQU %00000111 Clear all patterns from HM2007 * Result codes srTOOSLOW EQU $55 Voice was too long (too slow) srTOOFAST EQU $66 Voice was too short (too fast) srNOTFOUND EQU $77 Voice was not found ******************************************************************** * Constant definitions for general purpose * Port data direction registers (offset from $1000) REGBAS EQU $1000 DDRC EQU $07 Data direction for port C (0=Input, 1=Output) OC1M EQU $0C OC1 Action Mask register OC1D EQU $0D OC1 Action Data register TIC1 EQU $10 IC1 register (16 bits) TOC1 EQU $16 OC1 register (16 bits) TOC3 EQU $1A OC3 register (16 bits) TCTL1 EQU $20 OM2,OL2;OM3,OL3;OM4,OL4;OM5,OL5 TCTL2 EQU $21 -,-;EDG1B,EDG1A;EDG2B,EDG2A;EDG3B,EDG3A TMSK1 EQU $22 OC1I,OC2I,OC3I,OC4I,OC5I,IC1I,IC2I,IC3I TFLG1 EQU $23 OC1F,OC2F,OC3F,OC4F,OC5F,IC1F,IC2F,IC3F *PVIC1 EQU $00E8 EVB pseudo-vector for IC1 TMRREG EQU $0E Timer counter register (16 bits) * A/D controller and reading defines ADCTL EQU $30 ADR1 EQU $31 ADR2 EQU $32 ADR3 EQU $33 ADR4 EQU $34 Option EQU $39 **************************************************************************** * Voice clip indices * /* Voice response clips */ vLIKETOAST EQU %00000000 /* Would you like some toast? */ vTOAST EQU %00001100 /* toast */ vYES EQU %00010100 /* yes? */ vNO EQU %00011011 /* no */ vOKAY EQU %00100010 /* okay */ vHOWLIGHT EQU %00101001 /* how light? */ vLIGHT EQU %00110010 /* light */ vMEDIUM EQU %00111010 /* medium */ vDARK EQU %01000001 /* dark */ vUSING EQU %01001001 /* using setting *? vEND EQU %01010011 /* end */ vLOWERING EQU %01011011 /* lowering */ vRAISING EQU %01100011 /* raising */ vDONE EQU %01101011 /* done */ vSLOWER EQU %01110010 /* slower */ vFASTER EQU %01111010 /* faster */ vWHAT EQU %10000010 /* I'm sorry, what? */ vPLSSAY EQU %10001101 /* Please say -- */ vNOBREAD EQU %10010110 /* I should toast what? */ **************************************************************************** * Speech recognition indicies * /* Word Recognition indicies (bcd) */ wYES EQU $01 #define wYes '01' wNO EQU $02 #define wNo '02' wTOAST EQU $03 #define wToast '03' wLIGHT EQU $04 wMEDIUM EQU $05 wDARK EQU $06 wEND EQU $07 #define wEnd '07' wLASTWORD EQU $07 #define wLastWord '07' srMAXTRAIN EQU 3 #define srMaxTrain 3 srTHRESH EQU $80 #define srThresh 0x40 ********************************************************************************* * I/O port mapping for voice chip and motion detection VOCPORT EQU $1004 Port B isdENBAR EQU $1003 Port C port gPlayClip[0]; isdENMASK EQU %10000000 Bit 7 isdEOM EQU $1000 Port A port gEOM[0]; isdEOMMASK EQU %00000001 Bit 0 MOTPORT EQU $1000 Port A port gMotion[0]; MOTMASK EQU %00000010 Bit 1 ********************************************************************************* * Bread Platform pulse widths and control data OC3_OC1 EQU %00100000 OC1 controls A[5] OC3_CTL EQU %00100000 OC3 clears on compare PW_RAISE EQU $0D80 PW_LOWER EQU $0480 BREAD_DLY EQU 32 tLIGHT EQU $91 Treshold for light toast tMEDIUM EQU $93 Treshold for medium toast tDARK EQU $95 Treshold for dark toast ********************************************************************************* * Global Variables ORG $0048 * The only global variable. And it could even go away if I wanted to use * input capture. But I don't so it won't. 8) motLASTST RMB 1 Last known state of motion sensor srINITED RMB 1 1 == INIT_SR has been called, 0 otherwise TOASTLEVEL RMB 1 Toast level (threshold for temp IC) *********************************************************************** * Interrupt vectors, including RESET and IC1 ORG $FFFE RESET: jump to start of program FDB MAIN ORG $FFEE IC1 ISR FDB IC1ISR ORG $F800 * If we're not on the EVB, pass all the Buffalo functions to NULLFCN. NULLFCN RTS * BUFFALO I/O routines * Must be defined here because they must *follow* NULLFCN's definition * The $FFxx addresses are the addresses of the functions when used with the EVB OUTA EQU NULLFCN $FFB8 Output ASCII character in A OUT1BSP EQU NULLFCN $FFBE Output 1 byte in hex, followed by a space OUT1BYT EQU NULLFCN $FFBB Output 1 byte in hex OUTCRLF EQU NULLFCN $FFC4 Output carriage return and line feed OUTSTRG EQU NULLFCN $FFC7 Output string terminated by $04, incl. CR/LF OUTSTRG0 EQU NULLFCN $FFCA Output string terminated by $0F, excl. CR/LF *********************************************************************** * void main(void) * { * int w; MAIN LDS #$0047 JSR INIT Init(); LDAA #'M JSR OUTA JSR OUTCRLF * while(1) { MAINLOOP LDAA #'L * JSR OUTA * JSR OUTCRLF JSR WORDRECOG if (WordRecognized()) { BEQ ML1 JSR GETWORD w = GetWord(); PSHA TSX JSR OUT1BSP PULA SUBA #wTOAST if (w == wToast) BNE MAINLOOP JSR MAKETOAST MakeToast(); LDAA #'/ JSR OUTA LDAA #'P JSR OUTA BRA MAINLOOP * } ML1 JSR MOTDETECT else if (MotionDetected()) { BEQ MAINLOOP JSR RESPONDMOT RespondToMotion(); BRA MAINLOOP * } * } * } *********************************************************************** * void Init(void) * { INIT JSR INIT_VR InitVoiceResponse(); JSR INIT_SR InitSpeechRecognition(); JSR INIT_BREAD CLRA STAA motLASTST gLastMotState = 0; RTS * } **************************************************************************************** * void InitSpeechRecognition(void) * { INIT_SR LDAB #wLASTWORD int i=wLastWord; * int j; CLRA /* We're not fully initialized yet */ STAA srINITED JSR srCLEAR srClearPatterns(); * /* i == Accumulator B */ * /* j == Accumulator A */ * /* Train vocabulary */ ISR4 TBA while (i) { BEQ ISR1 LDAA #srMAXTRAIN j = srMaxTrain; ISR3 BEQ ISR2 while(j) { LDX #ISR_MSG1 puts("Please train word: "); PSHA JSR OUTSTRG ** TBA puts(words[i]); ** ADDA #'0 ** JSR OUTA LDX #srWORDS /* X <= &words[0].s */ PSHB DECB /* words is 0-based */ LSLB ABX /* X <= &words[0].s + 2*i */ LDX 0,X /* X <= words[i].s */ JSR OUTSTRG PULB LDAA #vPLSSAY JSR SAY LDX #vWORDS /* X <= &words[0].v */ ABX /* X == &words[i].v */ DEX /* words is 0-based */ LDAA 0,X /* A <= words[i].v */ JSR SAY TBA srTrainWord(i); JSR srTRAINWORD PULA j--; SUBA #1 BRA ISR3 } ISR2 SUBB #1 i--; BRA ISR4 } * /* Activate recognition circuit */ ISR1 LDAA #srRDSTATUS outs(srRdStatus); JSR OUTS ISR5 JSR INST while (inst() != srRdyCmd); SUBA #srRDYCMD BNE ISR5 LDAA #srWRINPUT outs(srWrInput); JSR OUTS LDAA #srCMDRECOG outk(srCmdRecog); JSR OUTK JSR INK ink(); LDAA #1 /* We're initialized now */ STAA srINITED RTS * } ************************************************************************************ * void InitVoiceResponse(void) * { INIT_VR LDX #$1000 BSET isdENBAR,X isdENMASK gISDEnableBar = 1; BSET DDRC,X isdENMASK /* Set pin as output */ BSET isdENBAR,X isdENMASK gISDEnableBar = 1; RTS * } ********************************************************************************* * bool WordRecognized(void) * { WORDRECOG LDAA #'W * JSR OUTA * JSR OUTCRLF LDAA #srRDSTATUS outs(srRdStatus); JSR OUTS JSR INST st = inst(); SUBA #srRDYCMD if (st == srRdyCmd) { BNE WR1 * JSR BEGINRECOG LDAA #1 return 1; RTS * } else WR1 CLRA return 0; RTS * } *************************************************************************** BEGINRECOG LDAA #'B * JSR OUTA * JSR OUTCRLF LDAA #srRDSTATUS outs(srRdStatus); JSR OUTS BR1 JSR INST while (inst() != srRdyCmd); SUBA #srRDYCMD BNE BR1 LDAA #srWRINPUT outs(srWrInput); JSR OUTS LDAA #srCMDRECOG outk(srCmdRecog); JSR OUTK JSR INK ink(); LDAA #20 BR2 NOP NOP SUBA #1 BNE BR2 LDAA #srRDSTATUS outs(srRdStatus); JSR OUTS JSR INST RTS ************************************************************************* * bcd GetWord(void) * { * bcd w; GETWORD LDAA #'G JSR OUTA * JSR OUTCRLF LDAA #srRDSTATUS outs(srRdStatus); JSR OUTS GW1 JSR INST while(inst() != srRdyCmd); SUBA #srRDYCMD BNE GW1 LDAA #srWRINPUT outs(srWrInput); JSR OUTS LDAA #srCMDRESULT outk(srCmdResult); JSR OUTK JSR INK ink(); * /* Read which word was read */ JSR READLOHI ReadLoHi(&w); PSHA TSX JSR OUT1BSP * /* Read score (required) and throw away */ JSR READLOHI ReadLoHi(NULL); PSHA TSX JSR OUT1BSP JSR BEGINRECOG PULB /* B <= score */ PULA /* A <= word */ SUBA #srTOOFAST BNE GW2 LDAA #vSLOWER JSR SAY BRA GETWORD GW2 ADDA #srTOOFAST SUBA #srTOOSLOW BNE GW3 LDAA #vFASTER JSR SAY BRA GETWORD GW3 ADDA #srTOOSLOW SUBA #srNOTFOUND BNE GW4 LDAA #vWHAT JSR SAY BRA GETWORD GW4 ADDA #srNOTFOUND /* A is word (again) */ PSHA TBA SUBA #srTHRESH BLO GW5 PULA LDAA #vWHAT JSR SAY BRA GETWORD GW5 LDAA #'/ JSR OUTA LDAA #'G JSR OUTA JSR OUTCRLF PULA RTS return w; * } ***************************************************************************** * void ReadLoHi(bcd *w) * { READLOHI LDAA #'R JSR OUTA LDAA #'L JSR OUTA LDAA #'H JSR OUTA JSR OUTCRLF LDAA #srRDSTATUS outs(srRdStatus); JSR OUTS RLH1 JSR INST while (inst() != srRdyLo); SUBA #srRDYLO BNE RLH1 LDAA #srRDOUTPUT outs(srRdOutput); JSR OUTS JSR INK w->lo = ink(); PSHA LDAA #%101 outs(101); JSR OUTS LDAA #%001 outs(001); JSR OUTS LDAA #%000 outs(000); JSR OUTS LDAA #srRDSTATUS outs(srRdStatus); JSR OUTS RLH2 JSR INST while(inst() != srRdyHi); SUBA #srRDYHI BNE RLH2 LDAA #srRDOUTPUT outs(srRdOutput); JSR OUTS JSR INK w->hi = ink(); LSLA LSLA LSLA LSLA PULB ABA PSHA LDAA #%101 outs(101); JSR OUTS LDAA #%001 outs(001); JSR OUTS LDAA #%000 outs(000); JSR OUTS LDAA #srRDSTATUS outs(srRdStatus); JSR OUTS JSR INST inst(); PULA RTS * } *********************************************************************************** * void srTrainWord(bcd index) * { srTRAINWORD PSHA /* Save index from register A */ LDAA #'T JSR OUTA LDAA #'W JSR OUTA TSX JSR OUT1BSP JSR OUTCRLF LDAA #srRDSTATUS outs(ReadStatus); JSR OUTS TW_1 JSR INST while(inst() != ReadyForCommand); SUBA #srRDYCMD BNE TW_1 LDAA #srWRINPUT outs(WriteInput); JSR OUTS LDAA #srCMDTRAIN outk(CmdTrain); JSR OUTK JSR INK LDAA #srRDSTATUS outs(ReadStatus); JSR OUTS TW_2 JSR INST while(inst() != ReadyForLo); PSHA SUBA #srRDYCMD BEQ TW_CONFZD PULA SUBA #srRDYLO BNE TW_2 LDAA #srWRINPUT outs(WriteInput); JSR OUTS PULA PSHA ANDA #$0F outk(loDigit); JSR OUTK JSR INK LDAA #srRDSTATUS outs(ReadStatus); JSR OUTS TW_3 JSR INST while(inst() != ReadyForHi); SUBA #srRDYHI BNE TW_3 LDAA #srWRINPUT outs(WriteInput); JSR OUTS PULA LSRA LSRA LSRA LSRA ANDA #$0F outk(hiDigit); JSR OUTK JSR INK LDAA #srRDSTATUS outs(ReadStatus); JSR OUTS TW_4 JSR INST inst(); SUBA #srRDYCMD while (inst() != ReadyCommand); /* Wait for word */ BNE TW_4 RTS TW_CONFZD PULA /* Restore the stack! */ LDX #CONFZD_MSG printf("Voice chip confused... trying again.\n"); JSR OUTSTRG PULA JMP srTRAINWORD * } ******************************************************************************** * void srClearPatterns(void) * { srCLEAR LDAA #srRDSTATUS outs(ReadStatus); JSR OUTS CLR_1 JSR INST while(inst() != ReadyForCommand); SUBA #srRDYCMD BNE CLR_1 LDAA #srWRINPUT JSR OUTS LDAA #srCMDRESET outk(CmdReset); JSR OUTK JSR INK ink(); LDAA #2000 CLR_2 NOP NOP SUBA #1 BNE CLR_2 LDAA #srRDSTATUS outs(ReadStatus); JSR OUTS JSR INST RTS * } **************************************************************************** * void MakeToast(void) * { * /* Check preconditions, like bread in toaster */ MAKETOAST LDAA #'M JSR OUTA LDAA #'T JSR OUTA JSR OUTCRLF JSR CHECKBREAD BNE MTOK LDAA #vNOBREAD JSR SAY RTS * /* Ask what toasting level */ MTOK LDAA #vHOWLIGHT JSR SAY MT0_1 JSR WORDRECOG BNE MT0_1 JSR GETWORD /* Light toast? */ SUBA #wLIGHT BNE MT0_2 LDAA #tLIGHT STAA TOASTLEVEL LDAB #vLIGHT BRA MTCONT MT0_2 ADDA #wLIGHT /* Dark toast? */ SUBA #wDARK BNE MT0_3 LDAA #tDARK STAA TOASTLEVEL LDAB #vDARK BRA MTCONT MT0_3 LDAA #tMEDIUM /* Medium toast (default) */ STAA TOASTLEVEL LDAB #vMEDIUM MTCONT PSHB LDAA #vUSING JSR SAY PULB TBA JSR SAY LDAA #vLOWERING Say(vLowering); JSR SAY JSR LOWER * while(1) { MT3 JSR CHECKTEMP if (TempThreshReached()) BNE MT4 break; JSR WORDRECOG if (!WordRecognized()) BEQ MT3 continue; JSR GETWORD if (GetWord() != wEnd) SUBA #wEND BNE MT3 continue; BRA MT4 break; * } MT4 LDAA #vRAISING Say(vRaising); JSR SAY JSR RAISE LDAA #vDONE Say(vDone); JSR SAY RTS * } **************************************************************************** * ***** CHRIS Read bread sensor here and put the number of slices loaded * ***** into accumulator A * For now, just lie and say that the toaster if fully loaded CHECKBREAD NOP * Set the ADCTL to read PE1 (left eye) LDX #REGBAS *** Eye 1 seems to be misbehaving. It is being software disabled for now. *** Misbehaving manifests itself by causing the microC to start executing *** instructions from anywhere within its memory, not nec. in PC order. **Eye1 LDAA #%10000001 ** STAA ADCTL,X ** JSR GetVal * ADR1 is one of 4 AD values we can use ** LDAA ADR1,X * I hope this constant works!!! ** SUBA #$80 ** BLS BreadIn1 ** CLRA ** BRA Eye2 **BreadIn1 LDAA #1 ** PSHA Eye2 LDAA #%10000010 STAA ADCTL,X JSR GetVal * ADR1 is one of 4 AD values we can use LDAA ADR1,X * I hope this constant works!!! SUBA #$80 BLS BreadIn2 CLRA ** PULA RTS BreadIn2 NOP ** PULA LDAA #1 RTS **************************************************************************** * ***** CHRIS Read the temperature IC. If the temperature is * above some threshold, return 1 in accumulator A. Else, return * 0 in A. For now, we'll lie and say that the temperature * has not been reached yet. CHECKTEMP NOP LDX #REGBAS * If counter is less than 25K, there's probably IC1 noise on the line. Don't * try to read the A/D port LDD TMRREG,X SUBD #25600 BLO CHKTEMP_END * Set the ADCTL to read PE7 LDX #REGBAS LDAA #%10000111 STAA ADCTL,X JSR GetVal * ADR1 is one of 4 AD values we can use LDAA ADR1,X STAA VOCPORT For debugging, output the value to the ISD's bus (port B) SUBA TOASTLEVEL BHS Hot CLRA RTS Hot LDAA #1 RTS CHKTEMP_END CLRA RTS *********************************************************** * Common ADC routine to wait until ADC is ready with value. GetVal NOP LDX #REGBAS WaitAD LDAA Option,X ANDA #%10000000 BEQ WaitAD ReadAD LDAA ADCTL,X ANDA #%10000000 BEQ ReadAD RTS **************************************************************************** * void Say(int vIndex) * { SAY STAA VOCPORT outva(vIndex); /* Output voice clip address */ LDAA #'S JSR OUTA JSR OUTCRLF LDX #$1000 gPlayClip = 0; BCLR isdENBAR,X isdENMASK LDAA #20 /* The low pulse must be long enough for the chip to catch it */ S0 NOP NOP SUBA #1 BNE S0 BSET isdENBAR,X isdENMASK gPlayClip = 1; BRSET isdEOM,X isdEOMMASK * while (!gEOM); BRCLR isdEOM,X isdEOMMASK * while (gEOM); /* low for 16 ms */ * /* If the toaster's voice triggered * the speech chip, reset the recognition */ LDAA #srRDSTATUS outs(srRdStatus); JSR OUTS JSR INST st = inst(); SUBA #srRDYCMD if (st != srRdyCmd) BNE S1 LDAA srINITED if (srInited) BEQ S1 JSR BEGINRECOG S1 RTS * } ********************************************************************************* * bool MotionDetected(void) * { MOTDETECT LDAA #'O * JSR OUTA * JSR OUTCRLF LDAA MOTPORT ANDA #MOTMASK BEQ MD1 if (gMotion) { LDAA motLASTST if (!gLastMotState) { BNE MD2 LDAA #1 gLastMotState = 1; STAA motLASTST RTS return 1; * } MD1 LDAA motLASTST } else if (gLastMotState) { BEQ MD2 CLRA STAA motLASTST gLastMotState = 0; * } * } MD2 CLRA RTS return 0; * } ********************************************************************************** * void RespondToMotion(void) * { RESPONDMOT LDAA #'R JSR OUTA LDAA #'M JSR OUTA JSR OUTCRLF LDAA #vLIKETOAST Say(vLikeToast); JSR SAY RM1 JSR WORDRECOG BNE RM1 JSR GETWORD /* Yes */ SUBA #wYES BNE RM2 JSR MAKETOAST RTS RM2 LDAA #vOKAY JSR SAY RTS * } ********************************************************************************* INIT_BREAD NOP * Set up IC1 to override microcontroller's grip on the servo * and raise the bread platform * These lines are needed to set up the IC1's ISR on the EVB * LDAA #$7E Jump (extended) opcode * STAA PVIC1 * LDX #IC1ISR * STX PVIC1+1 LDX #REGBAS * Set up OC3 (A[5]) to drop pulse at successful output compare BCLR TMSK1,X %11111000 BCLR TFLG1,X %11111000 Writing a 1 clears the bits LDAA #OC3_CTL STAA TCTL1,X LDD #PW_RAISE STD TOC3,X * Set up OC1 to raise pulse at timer overflow LDAA #OC3_OC1 OC1 affects OC3 STAA OC1M,X STAA OC1D,X OC1 writes a 1 to OC3 CLRA CLRB STD TOC1,X Successful compare at CLK == 0 BSET TMSK1,X %00000100 IC1 ISR BSET TFLG1,X %00000100 Clear IC1 flag (if any) LDAA #%00110000 Capture on any edge STAA TCTL2,X * Here's where we initialize the ADC. LDAA #%10000000 STAA Option,X CLI Enable interrupts ** JSR RAISE RTS ********************************************************************************* RAISE LDX #REGBAS ** LDAA #OC3_CTL Enable OC3 ** STAA TCTL1,X ** LDAA #OC3_OC1 Enable OC1 ** STAA OC1M,X LDD #PW_RAISE Raise bread platform STD TOC3+REGBAS ** JSR PAUSEBREAD ** CLRA ** STAA TCTL1,X Disable OC3 ** STAA OC1M,X Disable OC1 RTS ********************************************************************************* LOWER LDX #REGBAS ** LDAA #OC3_CTL Enable OC3 ** STAA TCTL1,X ** LDAA #OC3_OC1 Enable OC1 ** STAA OC1M,X LDD #PW_LOWER STD TOC3+REGBAS ** JSR PAUSEBREAD ** CLRA ** STAA TCTL1,X Disable OC3 ** STAA OC1M,X Disable OC1 RTS ********************************************************************************* IC1ISR LDX #REGBAS LDD #PW_RAISE STD TOC3,X BSET TFLG1,X %00000100 Clear IC1 flag RTI ********************************************************************************* PAUSEBREAD LDAA #BREAD_DLY PB1 PSHA JSR PAUSE32 PULA DECA BNE PB1 RTS **************************************************************************** INK LDX #$1000 Set direction bits to input BCLR DDRC,X #KMASK LDAA KBUS Read data from K bus ANDA #KMASK Discard non-K bus data RTS **************************************************************************** INST JSR INK Status register is on K bus ANDA #%00000011 Status bits are lower two RTS **************************************************************************** OUTK LDX #$1000 BSET DDRC,X #KMASK PSHA Save data to be written TSX LDAA KBUS Read current value ANDA #KMASK^$FF Save non-K bus part ORA 0,X Add K-bus part STAA KBUS Write data NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP PULA Clean up stack RTS **************************************************************************** OUTS LDX #$1000 BSET DDRC,X #SMASK LSLA 00000SSS -> 0000SSS0 LSLA 0000SSS0 -> 000SSS00 LSLA 000SSS00 -> 00SSS000 LSLA 00SSS000 -> 0SSS0000 PSHA Save data to be written TSX LDAA SBUS Read current value ANDA #SMASK^$FF Save non-S bus part ORA 0,X Add S-bus part STAA SBUS Write data NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP NOP PULA Clean up stack RTS PAUSE NOP NOP SUBA #1 BNE PAUSE RTS PAUSE32 LDAA #$FFFF P32 DECA BNE P32 RTS vWORDS FCB vYES,vNO,vTOAST,vLIGHT,vMEDIUM,vDARK,vEND srWORDS FDB srStrYES,srStrNO,srStrTOAST,srStrLIGHT,srStrMEDIUM,srStrDARK,srStrEND static char* words[] = { srStrYES FCC 'yes' "yes", FCB $04 srStrNO FCC 'no' "no", FCB $04 srStrTOAST FCC 'toast' "toast", FCB $04 srStrLIGHT FCC 'light' "light", FCB $04 srStrMEDIUM FCC 'medium' "medium", FCB $04 srStrDARK FCC 'dark' "dark", FCB $04 srStrEND FCC 'end' "end" FCB $04 * }; ISR_MSG1 FCC 'Train word: ' FCB $04 CONFZD_MSG FCC 'Voice chip confused... returning to command loop.' FCB $04
Once the green indicator lights and the toaster has said the word 'end', speak the word 'end' clearly to the toaster. Say the word exactly as you will say it later when you will ask for toast. In response, the toaster will ask you to say a word again, either the same word or another word. Each word will be said three times, and there are seven words. Once you have said the last word three times (the word 'yes'), the toaster enters its main loop.
To make toast
The toaster waits for either the user to say a word or for someone to walk
by. If someone walks by, the toaster asks that person, "Would you like
some toast?". If the person responds with 'no', the toaster will answer
"Okay" and re-enter the main loop. Else, if the person responds with
'yes', the toaster will begin its toast making process. This is the same
process as can be entered if the user asked the toaster for 'toast' from
the main loop.
At the start of the toast-making process, the toaster checks to see if there is bread in the toaster. If there is no bread, the toaster asks, "What should I toast?" and returns to the main command loop. Else, the toaster asks, "How light?", and expects a response of either 'light', 'medium', or 'dark'. If the toaster cannot understand what is said, it will take 'medium' as the default.
After a toast level has been set, the toaster will lower the bread platform and begin toasting. While toasting, the user may abort the process by saying 'end'. If the user does not abort, the toaster will raise the bread platform when the desired temperature threshold is met. After raising the bread platform, the toaster announces that it is done, and re-enters the command loop.
One note about using this toaster: after making one slice of toast, the toaster becomes very hot. Subsequent slices of toast will not be prepared fully as the first slice because the temperature sensor watches for a threshold, as opposed to differential temperature change. Thus, it is usually a good idea to let the toaster cool off for about five minutes between making batches of toast.
Corey's tasks
All of the VIS, including determining how the HM2007L speech recognition
system worked. Versions 0.1 through 1.6e2 of TOS. Wirewrapping of the
Main Controller Logic Board. Motion Sensor. Minor assembly of external
logic box.
Chris' tasks
Bread and temperature sensors. Bread Platform motor. Procurement of and
modifications to the toaster. TOS v1.4 and up. Major assembly of
external logic box. Soldering of the SDS Controller Board.
1 The "logic box" is a metal box with the same
footprint as the toaster. All circuitry is housed in this box to protect
the circuitry from the extreme temperatures of the toaster's heating
coils.
2 Of course, there is always the option of ordering food
to be delivered. But there still is the same fundamental problem: how to
do this without activating any more buttons or watching any more
lights.
3 There are two modes of operation for the HM2007L: CPU
mode and manual mode. In CPU mode, the chip obeys commands given on the
I/O bus. In manual mode, a keypad is connected to the I/O bus, and the
chip scans the keypad directly, obeying commands pressed by the user. The
distributor has constructed a test bed for the HM2007L in manual mode, but
has not done the same for CPU mode yet.
4 One degree Kelvin is equal to one degree Celcius. Zero
K = -273 deg C.
5 We had originally mounted two photoresistors, but due
to some bizarre A/D port trouble on the microcontroller, we were forced to
deactivate one of the photoresistors.
Corin Anderson |
corin@the4cs.com
Last modified: September 8, 1996