Sniffy the Virtual Rat: magazine training and shaping

Before beginning training with my real rat, Tessa, I was assigned the task of magazine training and shaping Sniffy the Virtual Rat. Sniffy is part of a computer program which simulates the real-world operant box environment, featuring a rat with variable behaviors, a bar which releases a pellet of virtual food into the food hopper when pressed, an indication light, and a water supply. Through this exercise, I was able to understand the characteristics of operant conditioning. This will allow me to correct any attentional or reinforcement errors on my behalf before attempting to train Tessa in the actual operant box.

---

Goal #1: To magazine train Sniffy the Virtual Rat in order for him to develop an association between the sound/bar and the reinforcement of virtual pellets in the virtual food hopper (magazine).

Procedure:
It took approximately 18 minutes to magazine train Sniffy. During the first three minutes of training, I allowed Sniffy to explore the cage while delivering reinforcement whenever he approached the hopper. The rate of reinforcement was increased as Sniffy approached the hopper. Repeated reinforcements were delivered to increase the association between sound/bar and reinforcement. Progress in building this association was displayed on a graph in the software, which allowed me to better estimate when and how to deliver reinforcement to Sniffy. Sniffy began to press the bar on his own after developing a 50% association between the sound/bar and reinforcement. The final ten minutes of training are characterized by a very high rate of reinforcement. Sniffy was reinforced almost immediately after receiving the previous reinforcement. This restricted him from wandering away from the food hopper, and the rate of association acquisition was increased.

Magazine Training Cumulative Record: Vertical dashes represent reinforcement delivered by the operator. Each region of the graph represents a five-minute period. "CRF Press Bar" indicates the beginning of Shaping. 

Goal #2: To shape sniffy to press the bar through the use of reinforcement. Shaping is achieved by reinforcing successive approximations of the bar-pressing behavior.

Procedure:
It took 55 minutes to successfully shape the bar-pressing behavior in my virtual rat. I began shaping by rewarding any rearing behaviors performed near the back of the cage. I initially rewarded Sniffy for sitting up and scratching his nose, but that did not lead to an increase in rearing behavior. Sniffy began pressing the bar after about five minutes of shaping. I continued to reinforce any rearing behaviors while tracking Sniffy's progress in building an association between bar pressing and reinforcement. I continued on an FR1 schedule of rearing behavior reinforcement until Sniffy began pushing the bar at least four times in a row. At this point, I observed Sniffy's behavior after reinforcement and reinforced rearing behavior on an FR3 schedule. Sniffy achieved a rate of about 20 bar-presses per five-minute period after 22 minutes of shaping. At this point, I allowed Sniffy to reinforce his own behavior by allowing the program to run on its own. Sniffy continued to press the bar at a rate of around 25 bar-presses per five-minute period for the final 33 minutes of shaping. I administered reinforcement whenever Sniffy wandered away from the magazine. This allowed the program to run a little quicker because bar-pressing behavior was more likely once Sniffy returned to the hopper for reinforcement.

Shaping Cumulative Record: Dashes on the top of the graph represent reinforcements administered by the operator. Dashes on the sloping line represent bar-presses by Sniffy. Click to enlarge. 

Sniffy was placed on a variable ratio schedule following shaping.