Training Sessions Five through Thirteen: Fixed Ratio Schedules of Reinforcement

Goal: 
To successfully train Tessa to press the bar for an increasing number of required behaviors. In order to reach the final behavior requirement of 20 bar-presses per reinforcement (FR 20), I gradually stretched the ratio from the original FR1 through FR2, FR3, FR5, FR7, FR10, and FR15.

Procedure: 
For each fixed ratio level, Tessa was reinforced for performing the correct number of bar-presses dictated by the FR level. Tessa was not manually reinforced for any behaviors, nor was she encouraged to approach the bar through additional reinforcement. Each session was run for a minimum of twenty minutes. Runtimes for each session are below:

Date	Schedule	Runtime (minutes)
9/25	FR2	23.75
9/26	FR3	21.58
9/27	FR3	22.00
9/28	FR5	21.50
9/29	FR7	22.67
9/30	FR10	28.52
10/01	FR15	21.77
10/02	FR15	24.00
10/04	FR20	29.33
10/05	FR20	22.26

 

Results:
Tessa performed a minimum of 200 bar-presses (FR2) and a maximum of 810 bar-presses (FR10) during her Fixed Ratio Training. FR3 and FR15 schedules were performed over multiple days to ensure Tessa learned the new schedule. As shown in Figure 1, Tessa performed more bar-presses on day 2 of the FR3 and day 2 of the FR15 schedules.

Figure 1: Total number of instrumental behaviors (bar-presses) per training session. Refer to Table 1 for session runtimes. Sessions are listed in the order they were performed.

Discussion:
Tessa made considerable progress through the Fixed Ratio Schedules of Reinforcement during her training. Initially, she became frustrated as the requirements for reinforcement were increased. For example, she began to exhibit escapist behaviors during the first day on FR2 schedules. These behaviors included climbing to the top of the Operant box on all sides, as well as actually climbing to the middle of the top of the Operant box. Although these behaviors were not eliminated, they were mitigated by blocking all exits from the box. You can read about how I reduced these behaviors here.

Although I was able to mitigate some of these troublesome behaviors, Tessa's wide variety of behaviors did affect the efficiency of her training. She was likely to perform a multitude of behaviors between each reinforcement, especially after at least ten minutes of training had elapsed. She would typically receive a reinforcement, then spend at least one minute exploring the rest of the Operant box. She would sniff the front window of the box, climb on all of the walls, and put her nose through any openings in the box. Nevertheless, she developed a better understanding of my expectations as training progressed. She was less likely to spend time exploring as the schedules were increased, which may be simply due to the higher requirements of each schedule. Furthermore, Tessa was likely to become distracted by any activity in the laboratory. This was worst when multiple students were in the lab at one time. In order to alleviate this, I used the rat privacy cover for FR10, FR15, and FR20 schedules.

On average, Tessa became satiated during each training session around 20 minutes of training. This lack of instrumental responses could be attributed to the fact that I ended her earliest shaping sessions after running less than 25 minutes of training. She may have expected to be released from the Operant box, and most importantly she may have assumed she would be released at a specific time and received rat chow in the holding room. The amount of rat chow given on the previous day did not seem to affect her motivation to perform in the Operant box. Likewise, I had difficult maintaining her target weight of 203 grams - however, she did not seem to be any less motivated when she gained weight compared to when she lost weight. You can read more about the schedule of food deprivation here.

Tessa's style of bar-pressing was not the desired style: instead of using her paws to press the bar, she typically pressed the bar with her nose or cheeks while holding a paw on the bar. This shows that the shaping of putting her paws on the bar was shaped successfully, but the behavior of pushing on the bar with her paws may have been overlooked - or possibly replaced by the reinforcement of pressing the bar with her cheeks during the FR schedules. Nevertheless, Dr. Trench once quoted B.F. Skinner as saying "a bar-press is a bar-press is a bar-press," so I did not attempt to change Tessa's style of bar-pressing.

The increase in Tessa's bar-pressing behavior was a function of the schedule she was on at the time: when she was on a higher schedule, she pressed the bar more times than on lower schedules. However, she did not achieve fifty reinforcements for each schedule of reinforcement. I thought fifty reinforcements would be a good goal, but she did not achieve this goal in either of the FR20 sessions. She received a maximum of 90 reinforcements (FR2) and an average of 68 reinforcements across all trials. Figure 2 illustrates the number of reinforcements received during each training session.

Figure 2. Total number of reinforcements per training session. Refer to Table 1 for training runtimes. Schedules are ordered by date they were performed.

Training Sessions Three and Four: Continuous Reinforcement of Bar-Pressing Behavior

Goal: 
To finalize the shaping process of Tessa the Rat until she was consistently pressing the bar on a continuous reinforcement schedule. Successive approximations of bar-pressing behaviors were reinforced, and Tessa was reinforced for each bar press.

Procedure: 
Training Session Three: Tessa was rewarded for any behavior resembling bar-pressing. This included rearing behaviors as well as approaches to the bar. Tessa was reinforced for placing her paws on the bar. This continued for a total of twenty minutes.

Training Session Four: Tessa was manually rewarded when she returned to the bar after leaving that side of the Operant box. She was reinforced for all bar-presses until she was consistently pressing the bar on an FR1 schedule. This continued for twenty-one minutes.

Results:
Training Session Three: Tessa received a total of 28 manual reinforcements for behaviors resembling bar-pressing. For example, Tessa was reinforced for approaching the bar from above but not from below. Likewise, rearing behaviors were only reinforced as she returned to her normal stance and crossed the bar threshold. Tessa pressed the bar 78 times, receiving a reinforcement for each bar-press.

Training Session Four: Tessa pressed the bar a total of 124 times and received a reinforcement for each bar-press. Manual reinforcements were used to encourage her to return to the bar after over thirty seconds of exploring other parts of the Operant box. Manual reinforcements were also given to encourage Tessa to approach the bar from above while not exploring the top of the cage (this was done by reinforcing rearing behavior as she returned to the height of the bar). She received a total of 6 manual reinforcements for such behaviors.

Discussion:
Training Session Three: Tessa became satiated quickly after receiving an adequate amount of reinforcements. This may have been caused by an increase in food intake from previous days of shaping when she performed at a much higher rate, received more reinforcements, and was fed an adequate amount of rat chow (around 5 grams each day in addition to consumed reinforcements). She performed at a rate of 30 bar-presses per five-minutes for the first ten minutes, then the rate dropped to zero by the end of twenty minutes.

Cumulative Record for Training Day Three

Training Session Four: Tessa developed a clear association between the bar-pressing behavior and the immediate reinforcement of the behavior. She performed at a high rate for the majority of the training session, but she became satiated during the last three minutes of training. I felt confident to stretch the ratio of Tessa's reinforcement schedule after observing her progress in the Operant box. A video of her behavior on the FR1 schedule is below:

This video illustrates Tessa's bar-pressing behavior as well as manual reinforcement of rearing behaviors as she returned to bar-pressing level.

Training Session Three was performed at 3:00 on September 22nd, 2012. 

Training Session Four was performed at 3:00 on September 24th, 2012.  

Training Session Two: Shaping Tessa

Goal
To shape Tessa the Rat until she forms an observable association between the act of pressing the bar in the Operant Box and the receipt of reinforcement in the food magazine. Shaping is the process of reinforcing successive approximations of the desired behavior until the organism is performing the desired response. 

Procedure
Food deprivation was continued to encourage Tessa to seek reinforcement in the Operant Box. At the time of training, Tessa weighed approximately 205 grams which was over her target weight of 202 grams. Training began at 3:00 PM in Operant Box 3 of the Learning Laboratory. Successive approximations of bar-pressing behavior were reinforced. First, Tessa was reinforced for turning or showing interest in the bar. Next, she was reinforced for moving towards the bar. Rearing behaviors were reinforced near the bar, and touching the bar was instantly reinforced. Variants of behavior resembling the target, but not leading to proper shaping, were not reinforced. For example, rearing behavior away from the bar were not reinforced; likewise, interest in the underside of the bar was not reinforced. The training session continued for 21:59, consisting of 134 manual reinforcements and 59 bar-presses.

Results
Tessa developed a strong association between the action of bar-pressing and the receipt of reinforcement. She was reinforced for multiple behaviors, but the final half of the training session was focused only on bar-pressing behaviors. Tessa responded to these reinforcements, and she was able to maintain steady bar-pressing behavior for nearly four minutes (see graph below).

Cumulative Record for Shaping

Discussion
Tessa's various forms of behavior have both positive and negative effects on the process of shaping. Because she produced a wide array of behaviors, it was not difficult to begin reinforcing the behaviors which resembled bar-pressing. However, once she developed an understanding of the contingency between bar-pressing and reinforcement, she continued to display a wide variety of other behaviors. These include rearing behaviors throughout the cage, interest in the lights above the food magazine and bar, and interest in the front of the cage and floor of the cage. Tessa received manual reinforcement when she lingered too long on one side of the cage. Rearing behaviors were reinforced after Tessa returned to bar level; thus, she was reinforced to stay at the bottom of the box instead of rearing or jumping towards the top. These reinforcements encouraged Tessa to return to the food magazine, which reminded her to engage in the bar-pressing activity.

Training day 2 began at 3:00 PM on September 21st, 2012.

Training Session One: Magazine Training Tessa

Goal: 
To magazine train (develop an association between the food hopper/magazine and the delivery of reinforcement in the Operant Box) Tessa the rat. My goal is to condition Tessa to consistently approach the food hopper after hearing the sound of the pellet dropping into the hopper and seeing the light in the box flash on/off.

Procedure:
My rat was deprived to approximately 87% of her original body weight before beginning training. On the training day, Tessa weighed approximately 200 grams, only one gram below her target weight. Training sessions are scheduled to begin at 3:00 PM each day, but may be delayed to 3:30 PM on Thursdays due to scheduling conflicts. On the first day of training, I worked with Dr. Trench to magazine train Tessa. I used the manual reinforcement trigger to deliver a pellet to the food magazine. Once Tessa ate the pellet, I immediately delivered another reinforcement. Reinforcements were delivered consistently for the duration of the training session. Magazine training was performed for 21 minutes, consisting of 69 manual reinforcements and 28 bar-press reinforcements. Shaping was roughly performed during this period to teach Tessa to associate bar-presses and the delivery of reinforcement.

Results:
Tessa consistently returned to the food magazine whenever reinforcement was delivered. She developed a strong association between the sound of pellet deliver, the flickering of the light, and delivery of reinforcement in the food magazine. She was also able to develop a slight association between the bar-pressing behavior and reinforcement.

Discussion:
Tessa displayed a wide variety of behaviors which allowed for some shaping to occur during the magazine training session. Consistent reinforcement delivery was important in maintaining Tessa's attention on the food magazine. The session was ended prematurely when Tessa appeared to be satiated. This occurred after the successful delivery of 69 manual reinforcements, as well as an astounding 28 bar-presses.

Magazine training of Tessa resembles magazine training as discussed in lecture. I found it easiest to reinforce Tessa immediately after receiving a reinforcement, and this seemed to encourage her to spend time around the magazine. Whenever she would travel away from the magazine, it was simple to get her to return: all I had to do was deliver another reinforcement. Soon, she was associating the sound of the food hopper with the delivery of reinforcement.

The shaping procedure will be further discussed in the discussion of Training Session Two.
  
Training Session One was performed on September 19th at 3:00 PM.  

Sniffy the Virtual Rat: magazine training and shaping

Before beginning training with my real rat, Tessa, I was assigned the task of magazine training and shaping Sniffy the Virtual Rat. Sniffy is part of a computer program which simulates the real-world operant box environment, featuring a rat with variable behaviors, a bar which releases a pellet of virtual food into the food hopper when pressed, an indication light, and a water supply. Through this exercise, I was able to understand the characteristics of operant conditioning. This will allow me to correct any attentional or reinforcement errors on my behalf before attempting to train Tessa in the actual operant box.

---

Goal #1: To magazine train Sniffy the Virtual Rat in order for him to develop an association between the sound/bar and the reinforcement of virtual pellets in the virtual food hopper (magazine).

Procedure:
It took approximately 18 minutes to magazine train Sniffy. During the first three minutes of training, I allowed Sniffy to explore the cage while delivering reinforcement whenever he approached the hopper. The rate of reinforcement was increased as Sniffy approached the hopper. Repeated reinforcements were delivered to increase the association between sound/bar and reinforcement. Progress in building this association was displayed on a graph in the software, which allowed me to better estimate when and how to deliver reinforcement to Sniffy. Sniffy began to press the bar on his own after developing a 50% association between the sound/bar and reinforcement. The final ten minutes of training are characterized by a very high rate of reinforcement. Sniffy was reinforced almost immediately after receiving the previous reinforcement. This restricted him from wandering away from the food hopper, and the rate of association acquisition was increased.

Magazine Training Cumulative Record: Vertical dashes represent reinforcement delivered by the operator. Each region of the graph represents a five-minute period. "CRF Press Bar" indicates the beginning of Shaping. 

Goal #2: To shape sniffy to press the bar through the use of reinforcement. Shaping is achieved by reinforcing successive approximations of the bar-pressing behavior.

Procedure:
It took 55 minutes to successfully shape the bar-pressing behavior in my virtual rat. I began shaping by rewarding any rearing behaviors performed near the back of the cage. I initially rewarded Sniffy for sitting up and scratching his nose, but that did not lead to an increase in rearing behavior. Sniffy began pressing the bar after about five minutes of shaping. I continued to reinforce any rearing behaviors while tracking Sniffy's progress in building an association between bar pressing and reinforcement. I continued on an FR1 schedule of rearing behavior reinforcement until Sniffy began pushing the bar at least four times in a row. At this point, I observed Sniffy's behavior after reinforcement and reinforced rearing behavior on an FR3 schedule. Sniffy achieved a rate of about 20 bar-presses per five-minute period after 22 minutes of shaping. At this point, I allowed Sniffy to reinforce his own behavior by allowing the program to run on its own. Sniffy continued to press the bar at a rate of around 25 bar-presses per five-minute period for the final 33 minutes of shaping. I administered reinforcement whenever Sniffy wandered away from the magazine. This allowed the program to run a little quicker because bar-pressing behavior was more likely once Sniffy returned to the hopper for reinforcement.

Shaping Cumulative Record: Dashes on the top of the graph represent reinforcements administered by the operator. Dashes on the sloping line represent bar-presses by Sniffy. Click to enlarge. 

Sniffy was placed on a variable ratio schedule following shaping.

Tessa's last snack and food deprivation progress

Tessa enjoys her last snack before the beginning of food deprivation in preparation for magazine training and shaping. 

Tessa loves her Cheerios - I used them to reward her after a successful handling session for two days in a row. She tends to hide her rat pellets under the sawdust when fed, but she immediately nibbles down any other snack! Since this video, Tessa has been deprived of food and reached her target weight of 203 grams in only four days! I will begin training Tessa on Wednesday. By the way, Tessa is now eating chocolate daily (chocolate sucrose pellets are used in our laboratory during training).

The current stats for Tessa's weight loss are below:

Target Weight = 203 Grams (87% Body Weight)

Date	Weight (grams)	Food Given (grams)
09/12/12	233	2.9
09/13/12	220	3.6
09/14/12	213.4	5.1
09/15/12	208	4.1
09/16/12	203	10
09/17/12	206	6

Early Handling Experiences with Tessa

Since meeting Tessa the Rat in lab last week, I have spent more than an hour handling and playing with her. She was extremely skittish around me from the beginning. I'm the tallest person in my class, so I was worried she would be a little more nervous around me than other students. On Thursday, I handled her for about five minutes before she finally calmed down. She loves to burrow between my elbow and torso, so I've found that to be an effective strategy in calming her down quickly. On Friday, I let her do that from the start and she calmed down almost instantly. She was pretty comfortable around me, and I figured out that she was more likely to cling to me when I took her away from her cage. When she is near the cage, she will try to squirm out of my arms and jump back in the cage; if I bring her to the other side of the room (even if she is near the classroom holding cage) she will calm down more quickly. She did bit me a few times on Friday, though, but I never put her down in response (I don't want to reinforce such behavior).

Earlier this week, I started feeding Tessa Cheerios after she calmed down during handling. She bit me a few times while trying to get her out of the cage, but that behavior stopped after I washed my hands. I am happy to say Tessa has gotten used to me over the past few days of handling! Now, she regularly calms down after under a minute of handling if I am sure to let her explore her surroundings. Dannele handled her for me recently and said she was very calm and easy-to-handle.