Assignment Schedules Of Reinforcement

For this worksheet, you will want to pay special attention to Chapter 7: Continuous vs Intermittent Schedules and the Four Basic Intermittent Schedules sections of the text.

Just want to post a quick note about the schedules of the reinforcement worksheet that is due this week. For this worksheet, you will want to pay special attention to Chapter 7: Continuous vs Intermittent Schedules and the Four Basic Intermittent Schedules sections of the text.

Schedules Of Reinforcement

Answer the following questions by identifying the correct schedule of reinforcement.

Provide A 50- To 75-Word Explanation of Why You Believe This Is the Schedule of Reinforcement Being Used.

  • Blass is tired of her students not being prepared for class each day. She knows that they are hard-working seniors who value getting an A in the class, and she wants to set up a schedule of work so they study hard and consistently. She schedules pop quizzes. Only she knows when those quizzes will be given. Assuming the students find the grade of A or B to be reinforcing, what schedule of reinforcement is at work here?
  • Sally broke up with her boyfriend. She refuses to text him back 90% of the time when he texts her. Sometimes, this means she will not text back after 10 or 20 texts. Finally, she gets frustrated (and lonely) and will text back but only to say ‘hi,’ ‘good luck at your football game,’ or other seemingly meaningless comments. This process continues for months, where he texts her a few times every day, and eventually she will break down and answer. Which schedule of reinforcement is the reason the boyfriend is still texting so long after the breakup?
  • Parents are teaching 2-year-old Johnny how to use the toilet. Every time he urinates even a little amount in the toilet they jump up and down, and he gets a bite of his favorite cookie. Which schedule of reinforcement is this an example of when first teaching a skill?
  • Manuel is thinking about opening a new coffee shop, which will compete with the big companies in town, but he is not sure how to draw in the crowds. To start, he offers reward points. When customers come in, they get a star for every $5 they spend. Once they have spent $50, they receive a free medium drip coffee. Which schedule of reinforcement is Manuel using to lure in repeat business?
  • Lee notices his patients are not coming in for their routine visits on time. They often push them back, which causes backups at some time in the year (before school starts) and low numbers at other times of year (summer). He wants more consistent clientele and work, so he offers a 10% discount on his current rates if patients stick to their defined three appointments a year (every 3.3 months). Patients motivated by this discount are adhering to which schedule of reinforcement?

CHAPTER 7

SCHEDULES AND THEORIES OF REINFORCEMENT

“I don’t understand why Alvin is so distant,” Mandy commented. “He was great when we started going out. Now it’s like pulling teeth to get him to pay attention to me.” “So why do you put up with it?” her sister asked.

“I guess I’m in love with him. Why else would I be so persistent?”

Schedules of Reinforcement

In this section, we discuss schedules of reinforcement. A schedule of reinforcement is the response requirement that must be met to obtain reinforcement. In other words, a schedule indicates what exactly has to be done for the reinforcer to be delivered. For example, does each lever press by the rat result in a food pellet or are several lever presses required?

Did your mom give you a cookie each time you asked for one, or only some of the time? And just how persistent does Mandy have to be before Alvin will pay attention to her? As you will discover in this section, different response requirements can have dramatically different effects on behavior. Many of these effects (known as schedule effects) were first observed in experiments with pigeons (Ferster & Skinner, 1957), but they also help explain some puzzling aspects of human behavior that are often attributed to internal traits or desires.

Continuous Versus Intermittent Schedules

A continuous reinforcement schedule is one in which each specified response is reinforced. For example, each time a rat presses the lever, it obtains a food pellet; each time the dog rolls over on command, it gets a treat; and each time Karen turns the ignition in her car, the motor starts. Continuous reinforcement (abbreviated CRF) is very useful when a behavior is first being shaped or strengthened. 

For example, when using a shaping procedure to train a rat to press a lever, reinforcement should be delivered for each approximation to the target behavior. Similarly, if we wish to encourage a child to always brush her teeth before bed, we would do well to initially praise her each time she does so.

An intermittent (or partial) reinforcement schedule is one in which only some responses are reinforced. For example, perhaps only some of the rat’s lever presses result in a food pellet, and perhaps only occasionally did your mother give you a cookie when you asked for one. Intermittent reinforcement obviously characterizes much of everyday life.

Not all concerts we attend are enjoyable, not every person we invite out on a date accepts, and not every date that we go out on leads to an enjoyable evening. And although we might initially praise a child each time she properly completes her homework, we might soon praise her only occasionally in the belief that such behavior should persist in the absence of praise.

There are four basic (or simple) types of intermittent schedules: fixed ratio, variable ratio, fixed interval, and variable interval. We will describe each one along with the characteristic response pattern produced by each. Note that this characteristic response pattern is the stable pattern that emerges once the organism has had considerable exposure to the schedule. Such stable patterns are known as steady-state behaviors, in contrast to the more variable patterns of behavior that are evident when an organism is first learning a schedule.

QUICK QUIZ A

  1. A s________________ of reinforcement is the r________________ requirement that must be met in order to obtain reinforcement.
  2. On a c________________ reinforcement schedule (abbreviated ________________), each response is reinforced, whereas on an i________________ reinforcement schedule, only some responses are reinforced. The latter is also called a p________________ reinforcement schedule.
  3. Each time you flick the light switch, the light comes on. The behavior of flicking the light switch is on a(n) ________________ schedule of reinforcement.
  4. When the weather is very cold, you are sometimes unable to start your car. The behavior of starting your car in very cold weather is on a(n) ________________ schedule of reinforcement.
  5. S________________ e________________ are the different effects on behavior produced by different response requirements. These are the stable patterns of behavior that emerge once the organism has had sufficient exposure to the schedule. Such stable patterns are known as st________________-st________________ behaviors.

Four Basic Intermittent Schedules

Fixed Ratio Schedules On a fixed ratio (FR) schedule, reinforcement is contingent upon a fixed, predictable number of responses. For example, on a fixed ratio 5 schedule

(abbreviated FR 5), a rat has to press the lever 5 times to obtain a food pellet. On an FR 50 schedule, it has to press the lever 50 times to obtain a food pellet. Similarly, earning a dollar for every 10 carburetors assembled on an assembly line is an example of an FR

10 schedules, while earning a dollar for each carburetor assembled is an example of an FR 1 schedule. Note that an FR 1 schedule is the same as a CRF (continuous reinforcement) schedule in which each response is reinforced (thus, such a schedule can be correctly labeled as either an FR 1 or a CRF).

FR schedules generally produce a high rate of response along with a short pause following the attainment of each reinforcer (see Figure 7.1). This short pause is known as a post-reinforcement pause. For example, a rat on an FR 25 schedule will rapidly emit 25 lever presses, munch down the food pellet it receives, and then snoop around the chamber for a few seconds before rapidly emitting another 25 lever presses. 

In other words, it will take a short break following each reinforcer, just as you might take a short break after reading each chapter in a textbook or completing a particular assignment. Note, too, that each pause is usually followed by a relatively quick return to a high rate of response. Thus, the typical FR pattern is described as a “break-and-run” pattern—a short break followed by a steady run of responses.

FIGURE 7.1 Response patterns for fixed ratio (FR), variable ratio (VR), fixed interval (Fl), and variable interval (VI) schedules. This figure shows the characteristic pattern of responding on each of the four basic schedules. Notice the high response rate on the FR and VR schedules, moderate response rate on the VI schedule, and scalloped response pattern on the FI schedule. Also, both the FR and FI schedules are accompanied by post-reinforcement pauses.

In general, higher ratio requirements produce longer post-reinforcement pauses. This means that you will probably take a longer break after completing a long assignment than after completing a short one. Similarly, a rat will show longer pauses on an FR 100 schedule than on an FR 30 schedule. With very low ratios, such as CRF or FR 2, there may be little or no pausing other than the time it takes for the rat to munch down the food pellet. In such cases, the next reinforcer is so close—only a few levers pressed away—that the rat is tempted to immediately go back to work. (If only the reinforcers for studying were so immediate!)

Schedules in which the reinforcer is easily obtained are said to be very dense or rich, while schedules in which the reinforcer is difficult to obtain are said to be very lean. Thus, an FR 5 schedule is considered a very dense schedule of reinforcement compared to an FR 100. During a 1-hour session, a rat can earn many more food pellets on an FR 5 schedule than it can on an FR 100. Likewise, an assembly line worker who earns a dollar for each carburetor assembled (a CRF schedule) is able to earn considerably more during an 8-hour shift than is a worker who earns a dollar for every 10 carburetors assembled (an FR 10 schedule).

In general, “stretching the ratio”—moving from a low ratio requirement (a dense schedule) to a high ratio requirement (a lean schedule)—should be done gradually. For example, once lever pressing is well established on a CRF schedule, the requirement can be gradually increased to FR 2, FR 5, FR 10, and so on. If the requirement is increased too quickly—for example, CRF to FR 2 and then a sudden jump to FR 20—the rat’s behavior may become erratic and even die out altogether. Likewise, if you try to raise the requirement too high—say, to FR 2000—there may be a similar breakdown in the rat’s behavior. Such breakdowns in behavior are technically known as ratio strain, which is a disruption in responding due to an overly demanding response requirement.

Ratio strain is what most people would refer to as burnout, and it can be a big problem for students faced with a heavy workload. Some students, especially those who have a history of getting by with minimal work, may find it increasingly difficult to study under such circumstances and may even choose to drop out of college. If they had instead experienced a gradual increase in workload over a period of several months or years, they might have been able to put forth the needed effort to succeed.

QUICK QUIZ B

  1. On a(n) ________________ schedule, reinforcement is contingent upon a fixed number of responses.
  2. A schedule in which 15 responses are required for each reinforcer is abbreviated ________________.
  3. A mother finds that she always has to make the same request three times before her child complies. The mother’s behavior of making requests is on a(n) ________________ schedule of reinforcement.
  4. An FR 1 schedule of reinforcement can also be called a ________________ schedule.
  5. A fixed ratio schedule tends to produce a (high/low) rate of response, along with a p________________ p________________.
  6. An FR 200 schedule of reinforcement will result in a (longer/shorter) pause than an FR 50 schedule.
  7. The typical FR pattern is sometimes called a b________________-and-r________________ pattern, with a ________________ pause that is followed immediately by a (high/low) rate of response.
  8. An FR 12 schedule of reinforcement is (denser/leaner) than an FR 75 schedule.
  9. A dense schedule of reinforcement can also be referred to as a r________________ schedule.
  10. Over a period of a few months, Aaron changed from complying with each of his mother’s requests to complying with every other request, then with every third request, and so on. The mother’s behavior of making requests has been subjected to a procedure known as “s________________ the r________________.”
  11. Graduate students often have to complete an enormous amount of work in the initial year of their program. For some students, the workload involved is far beyond anything they have previously encountered. As a result, their study behavior may become increasingly (erratic/stereotyped) throughout the year, a process known as r________________ s________________.

Study Tip: Post-reinforcement pauses may have important implications for studying. On the one hand, they suggest that there may be a natural tendency for us to take a break after completing a certain amount of work. On the other hand, when students take a break from studying, they often have difficulty starting up again.

A possible explanation is that students, unlike pigeons and rats who have little to do during a post-reinforcement pause, tend to engage in highly reinforcing activities during a study break, such as playing computer games or watching television. A possible solution to this problem might be to ensure that study breaks, especially the ones you intend to be brief, do not involve anything particularly interesting; for example, rather than watching television, you might instead do some light housework, a bit of exercise, or perhaps just sit and listen to some quiet music.

Like a pigeon with a break-and-run pattern, you may now find it much easier to resume studying. As for more reinforcing activities, such as TV watching, these can be saved for the end of the study session, where you can partake of them as major reinforcers for a job well done (though, as you will learn in Chapter 10, there are questions concerning the extent to which such “self-reinforcement” constitutes real reinforcement).

Another useful trick, which can be used whenever you have difficulty sitting down to study, is the just-get-started (JGS) tactic (e.g., Pychyl, 2010). In this case, you set yourself a very small and easily attained goal, such as to study for at least 5 or 10 minutes with the option to then quit if you still “don’t feel like studying.” Many students find that once the allotted time has passed, it is surprisingly easy to carry on—in the same way that once a pigeon starts responding on an FR schedule, it has a strong tendency to complete the entire run of responses up to the next reinforcer.

Many people report finding the JGS tactic to be very helpful in reducing the tendency to procrastinate (such as when working on term papers), and you may wish to experiment with it yourself.

Variable Ratio Schedules On a variable ratio (VR) schedule, reinforcement is contingent upon a varying, unpredictable number of responses. For example, on a variable ratio 5 (VR 5) schedule, a rat has to emit an average of 5 lever presses for each food pellet, with the number of lever responses on any particular trial varying between, say, 1 and 10. 

Thus, the number of required lever presses might be 3 for the first pellet, 6 for the second pellet, 1 for the third pellet, 7 for the fourth pellet, and so on, with the overall average being 5 lever presses for each reinforcer. Similarly, on a VR 50 schedule, the number of required lever presses may vary between 1 and 100, with the average being 50.

VR schedules generally produce a high and steady rate of response, often with little or no post-reinforcement pause (see Figure 7.1). A post-reinforcement pause is especially unlikely to occur when the minimum response requirement in the schedule is very low, such as a VR 50 schedule in which the requirement varies between 1 and 100 as opposed to a VR 50 in which the requirement varies between, say, 10 and 90

(Schlinger, Derenne, & Baron, 2008). This is understandable if you consider that each response on the former schedule has the potential of resulting in a reinforcer even if a reinforcer has just been obtained.

The real world is filled with examples of VR schedules. Some predatory behaviors, such as that shown by cheetahs, have a strong VR component in that only some attempts at chasing down prey are successful. In humans, only some acts of politeness receive an acknowledgment, only some residents who are called upon by canvassers will make a contribution, and only some TV shows are enjoyable.

Many sports activities, such as shooting baskets in basketball and shots on goal in hockey, are also reinforced largely on a VR schedule. As I am writing this passage, a colleague stopped by and joked that his golf drive is probably on a VR 200 schedule. In other words, he figures that an average of about 1 in every 200 drives is a good one. I (Russ Powell) replied that my own drives are on a much leaner schedule with the result that ratio strain has set in, which is fancy behaviorist talk for “I so rarely hit the ball straight that I have just about given up playing.”

Variable ratio schedules help to account for the persistence with which some people display certain maladaptive behaviors. Gambling is a prime example in this regard: The unpredictable nature of these activities results in a very high rate of behavior. In fact, the behavior of a gambler playing a slot machine is the classic example of a VR schedule in humans. Certain forms of aberrant social behavior may also be accounted for by VR schedules.

For example, why do some men persist in using cute, flippant remarks to introduce themselves to women when the vast majority of women view such remarks negatively? One reason is that a small minority of women actually respond favorably, thereby intermittently reinforcing the use of such remarks. For example, Kleinke, Meeker, and Staneske (1986) found that although 84% of women surveyed rated the opening line “I’m easy. Are you?” as poor to terrible, 14% rated it as either very good or excellent!

Variable ratio schedules of reinforcement may also facilitate the development of an abusive relationship. At the start of a relationship, the individuals involved typically provide each other with an enormous amount of positive reinforcement (a very dense schedule). This strengthens the relationship and increases each partner’s attraction to the other. 

As the relationship progresses, such reinforcement naturally becomes somewhat more intermittent. In some situations, however, this process becomes malignant, with one person (let us call this person the victimizer) providing reinforcement on an extremely intermittent basis, and the other person (the victim) working incredibly hard to obtain that reinforcement. Because the process evolves gradually (a process of slowly “stretching the ratio”), the victim may have little awareness of what is happening until the abusive pattern is well established.

What would motivate such an unbalanced process? One source of motivation is that the less often the victimizer reinforces the victim, the more attention (reinforcement) he or she receives from the victim. In other words, the victim works so hard to get the partner’s attention that he or she actually reinforces the very process of being largely ignored by that partner.

Of course, it does not necessarily have to be a one-way process, and there may be relationships in which the partners alternate the role of victim and victimizer. The result may be a volatile relationship that both partners find exciting but that is constantly on the verge of collapse due to frequent periods in which each partner experiences “ratio strain.”

QUICK QUIZ C

  1. On a variable ratio schedule, reinforcement is contingent upon a ________________un________________ ________________ of responses.
  2. A variable ratio schedule typically produces a (high/low) rate of behavior (with/without) a post-reinforcement pauses.
  3. An average of 1 in 10 people approached by a panhandler actually gives him money. His behavior of panhandling is on a(n) ________________ (be precise and use the abbreviation) schedule of reinforcement.
  4. As with an FR schedule, an extremely lean VR schedule can result in r________________ s________________.

Fixed Interval Schedules On a fixed interval (FI) schedule, reinforcement is contingent upon the first response after a fixed, predictable period of time. For a rat on a fixed interval 30-second (FI 30-sec) schedule, the first lever press after a 30-second interval has elapsed results in a food pellet. Following that, another 30 seconds must elapse before a lever press will again produce a food pellet. Any lever pressing that occurs during the interval, before the 30-second period has elapsed, is ineffective. Similarly, trying to phone a business that opens in exactly 30 minutes will be effective only after the 30 minutes have elapsed, with any phone calls before that being ineffective.

FI schedules often produce a “scalloped” (upwardly curved) pattern of responding, consisting of a post-reinforcement pause followed by a gradually increasing rate of response as the interval draws to a close (see Figure 7.1). For example, a rat on an FI a 30-sec schedule will likely emit no lever presses at the start of the 30-second interval. 

This will be followed by a few tentative levers pressed perhaps midway through the interval, with a gradually increasing rate of response thereafter. By the time the interval draws to a close and the reinforcer is imminent, the rat will be emitting a high rate of response, with the result that the reinforcer will be attained as soon as it becomes available.

Would the behavior of trying to phone a business that opens in 30 minutes also follow a scalloped pattern? If we have a watch available, it probably would not. We would simply look at our watch to determine when the 30 minutes had elapsed and then make the phone call. The indicated time would be a discriminative stimulus (SD) for when the reinforcer is available (i.e., the business is open), and we would wait until the appropriate time before phoning.

But what about the behavior of looking at your watch during the 30 minutes (the reinforcer for which would be noticing that the interval has elapsed)? You are unlikely to spend much time looking at your watch at the start of the interval. As time progresses, however, you will begin looking at it more and more frequently. In other words, your behavior will follow the typical scalloped pattern of responding.

The distribution of study sessions throughout the term can also show characteristics of an FI scallop, which can again contribute to students’ tendency to procrastinate in their studying. At the start of a course, many students engage in little or no studying, given that the first exam is some distance away. This is followed by a gradual increase in studying as the first exam approaches.

The completion of the exam is again followed by little or no studying until the next exam approaches. Unfortunately, these post-reinforcement pauses are often too long, with the result that many students obtain much poorer marks than they would have if they had studied at a steadier pace throughout. (Note, however, that studying for exams is not a pure example of an FI schedule, because a certain amount of work must be accomplished during the interval to obtain the reinforcer of a good mark. On a pure FI schedule, any response that happens during the interval is essentially irrelevant.)

QUICK QUIZ D

  1. On a fixed interval schedule, reinforcement is contingent upon the ________________ response following a(n) ________________, pr ________________ period of ________________.
  2. If I have just missed the bus when I get to the bus stop, I know that I have to wait 15 minutes for the next one to come along. Given that it is absolutely freezing out, I snuggle into my parka as best I can and grimly wait out the interval. Every once in a while, though, I emerge from my cocoon to take a quick glance down the street to see if the bus is coming. My behavior of looking for the bus is on a(n) ________________ (use the abbreviation) schedule of reinforcement.
  3. In the example in question 2, I will probably engage in (few/frequent) glances at the start of the interval, followed by a gradually (increasing/decreasing) rate of glancing as time passes.
  4. Responding on an FI schedule is often characterized by a sc________________ pattern of responding consisting of a p________________ p________________ followed by a gradually (increasing/decreasing) rate of behavior as the interval draws to a close.
  5. On a pure FI schedule, any response that occurs (during/following) the interval is irrelevant.

Variable Interval Schedules On a variable interval (VI) schedule, reinforcement is contingent upon the first response after a varying, unpredictable period of time. For a rat on a variable interval 30-second (VI 30-sec) schedule, the first lever press after an average interval of 30 seconds will result in a food pellet, with the actual interval on any particular trial varying between, say, 1 and 60 seconds.

Thus, the number of seconds that must pass before a lever press will produce a food pellet could be 8 seconds for the first food pellet, 55 seconds for the second pellet, 24 seconds for the third, and so on, the average of which is 30 seconds. Similarly, if each day you are waiting for a bus and have no idea when it will arrive, then looking down the street for the bus will be reinforced after a varying, unpredictable period of time—for example, 2 minutes the first day, 12 minutes the next day, 9 minutes the third day, and so on, with an average interval of, say, 10 minutes (VI 10-min).

VI schedules usually produce a moderate, steady rate of response, often with little or no post-reinforcement pause (see Figure 7.1). By responding at a relatively steady rate throughout the interval, the rat on a VI 30-sec schedule will attain the reinforcer almost as soon as it becomes available. 

Similarly, if you need to contact your professor with a last-minute question about an assignment and know that she always arrives in her office sometime between 8:00 A.M. and 8:30 A.M., a good strategy would be to phone every few minutes throughout that time period. By doing so, you will almost certainly contact her within a few minutes of her arrival.

Because VI schedules produce steady, predictable response rates, they are often used to investigate other aspects of operant conditioning, such as those involving choice between alternative sources of reinforcement. You will encounter examples of this when we discuss choice behavior in Chapter 10.

QUICK QUIZ E

  1. On a variable interval schedule, reinforcement is contingent upon the ________________ response following a ________________, un ________________ period of ________________.
  2. You find that by frequently switching stations on your radio, you are able to hear your favorite song an average of once every 20 minutes. Your behavior of switching stations is thus being reinforced on a ________________ schedule.
  3. In general, variable interval schedules produce a (low/moderate/high) and (steady/fluctuating) rate of response with little or no ________________ ________________.

Comparing the Four Basic Schedules The four basic schedules produce quite different patterns of behavior, which vary in both the rate of response and in the presence or absence of a post-reinforcement pause. These characteristics are summarized in Table 7.1. As can be seen, ratio schedules (FR and VR) produce higher rates of response than do interval schedules (FI and VI). This makes sense because the reinforcer in ratio schedules is entirely “response contingent”; that is, it depends entirely on the number of responses emitted. 

A rat on a VR 100 schedule can double the number of food pellets earned in a 1-hour session by doubling its rate of lever pressing. Similarly, a door-to-door salesman can double the number of sales he makes during a day by doubling the number of customers he calls on (assuming that he continues to give an adequate sales pitch to each customer).

Compare this to an interval schedule in which reinforcement is mostly time contingent. For example, on an FI 1-minute schedule, no more than 50 reinforcers can be earned in a 50-minute session. Under such circumstances, responding at a high rate throughout each interval does not pay off and is essentially a waste of energy. Instead, it makes more sense to respond in a way that will simply maximize the possibility of attaining the reinforcer soon after it becomes available. On an FI schedule, this means responding at a gradually increasing rate as the interval draws to a close; on a VI schedule, this means responding at a moderate, steady pace throughout the interval.

TABLE 7.1 Characteristic response rates and post-reinforcement pauses for each of the four basic intermittent schedules. These are only general characteristics; they are not found under all circumstances. For example, an FR schedule with a very low response requirement, such as FR 2, is unlikely to produce a post-reinforcement pause. By contrast, an FR schedule with a very high response requirement, such as FR 2000, may result in ratio strain and a complete cessation of responding.

 FRVRFIVI
Response rateHighHighIncreasingModerate
Post-reinforcement pauseYesNoYesNo

It can also be seen that fixed schedules (FR and FI) tend to produce post-reinforcement pauses, whereas variable schedules (VR and VI) do not. On a variable schedule, there is often the possibility of a relatively immediate reinforcer, even if one has just obtained a reinforcer, which tempts one to immediately resume responding.

By comparison, on a fixed schedule, attaining one reinforcer means that the next reinforcer is necessarily some distance away. On an FR schedule, this results in a short post-reinforcement pause before grinding out another set of responses; on an FI schedule, the post-reinforcement pause is followed by a gradually increasing rate of response as the interval draws to a close and the reinforcer becomes imminent.

Which of these workers is on a ratio schedule of reinforcement?

QUICK QUIZ F

  1. In general, (ratio/interval) schedules tend to produce a high rate of response. This is because the reinforcer in such schedules is entirely r________________ _ contingent, meaning that the rapidity with which responses are emitted (does/does not) affect how soon the reinforcer is obtained.
  2. On ________________ schedules, the reinforcer is largely time contingent, meaning that the rapidity with which responses are emitted has (little/considerable) effect on how soon the reinforcer is obtained.
  3. In general, (variable/fixed) schedules produce little or no post-reinforcement pausing because such schedules often provide the possibility of relatively i________________ reinforcement, even if one has just obtained a reinforcer.
  4. In general, ________________ _ schedules produce post-reinforcement pauses because obtaining one reinforcer means that the next reinforcer is necessarily quiet (distant/near).

Other Simple Schedules Of Reinforcement

Duration Schedules On a duration schedule, reinforcement is contingent on performing a behavior continuously throughout a period of time. On a fixed duration (FD) schedule, the behavior must be performed continuously for a fixed, predictable period of time. For example, the rat must run in the wheel for 60 seconds to earn one pellet of food (an FD 60-sec schedule). Likewise, Julie may decide that her son can watch television each evening only after he completes 2 hours of studying (an FD 2-hr schedule).

On a variable duration (VD) schedule, the behavior must be performed continuously for a varying, unpredictable period of time. For example, the rat must run in the wheel for an average of 60 seconds to earn one pellet of food, with the required time varying between 1 second and 120 seconds on any particular trial (a VD 60-sec schedule). And Julie may decide to reinforce her son’s studying with cookies and other treats at varying points in time that happen to average out to about one treat every 30 minutes (a VD 30-min schedule). (Question: How do FD and VD schedules differ from FI and VI schedules)?

Although duration schedules are sometimes useful in modifying certain human behaviors, such as studying, they are in some ways rather imprecise compared to the four basic schedules discussed earlier. With FR schedules, for example, one knows precisely what was done to achieve the reinforcer, namely, a certain number of responses. On an FD schedule, however, what constitutes “continuous performance of behavior” during the interval could vary widely.

With respect to wheel running, for example, a “lazy” rat could dawdle along at barely a walk, while an “energetic” rat might rotate the wheel at a tremendous pace. Both would receive reinforcements. Similarly, Julie’s son might read only a few pages during his 2-hour study session or charge through several chapters; in either case, he would receive the reinforcer of being allowed to watch television.

Remember too, from Chapter 6, how reinforcing the mere performance of an activity with no regard to level of performance can undermine a person’s intrinsic interest in that activity. This danger obviously applies to duration schedules; one therefore needs to be cautious in their use. Nevertheless, many successful writers write according to an FD schedule of reinforcement (i.e., they write for a fixed amount of time each day), although there are also some who write according to an FR schedule—that is, they strive to complete a certain number of words or pages each day (Currey, 2013; Silvia, 2007). An informal survey of your fellow students will likely show that they too vary in the extent to which they study for time or to accomplish a certain amount of work (e.g., to finish a chapter), something you may wish to explore in optimizing your own study behavior.

Response-Rate Schedules As we have seen, different types of intermittent schedules produce different rates of response (i.e., they have different schedule effects). These different rates are essentially by-products of the schedule. However, in a response-rate schedule, reinforcement is directly contingent upon the organism’s rate of response. Let’s examine three types of response-rate schedules.

In differential reinforcement of high rates (DRH), reinforcement is contingent upon emitting at least a certain number of responses in a certain period of time—or, more generally, reinforcement is provided for responding at a fast rate. The term differential reinforcement means simply that one type of response is reinforced while another is not. In a DRH schedule, reinforcement is provided for a high rate of response and not for a low rate.

For example, a rat might receive a food pellet only if it emits at least 30 lever presses within a period of a minute. Similarly, a worker on an assembly line may be told that she can keep her job only if she assembles a minimum of 20 carburetors per hour. By requiring so many responses in a short period of time, DRH schedules ensure a high rate of responding. Athletic events such as running and swimming are prime examples of DRH schedules in that winning is directly contingent on a rapid series of responses.

In differential reinforcement of low rates (DRL), a minimum amount of time must pass between each response before the reinforcer will be delivered—or, more generally, reinforcement is provided for responding at a slow rate. For example, a rat might receive a food pellet only if it waits at least 10 seconds between lever presses. So how is this different from an FI 10-sec schedule?

Remember that on an FI schedule, responses that occur during the interval have no effect; on a DRL schedule, however, responses that occur during the interval do have an effect—an adverse effect in that they prevent reinforcement from occurring. In other words, responding during the interval must not occur in order for a response following the interval to produce a reinforcer.

Human examples of DRL schedules consist of situations in which a person is required to perform an action slowly. For example, a parent might praise a child for brushing her teeth slowly or completing her homework slowly, given that going too fast generally results in sloppy performance. Once the quality of performance improves, reinforcement can then be made contingent on responding at a normal speed.

In differential reinforcement of paced responding (DRP), reinforcement is contingent upon emitting a series of responses at a set rate—or, more generally, reinforcement is provided for responding neither too fast nor too slow. For example, a rat might receive a food pellet if it emits 10 consecutive responses, with each response separated by an interval of no less than 1.5 and no more than 2.5 seconds. Similarly, musical activities, such as playing in a band or dancing to music, require that the relevant actions be performed at a specific pace. People who are very good at this are said to have a good sense of timing or rhythm. Further examples of DRP schedules can be found in noncompetitive swimming or running.

People often perform these activities at a pace that is fast enough to ensure benefits to health and a feeling of well-being, yet not so fast as to result in exhaustion and possible injury. In fact, even competitive swimmers and runners, especially those who compete over long distances, will often set a specific pace throughout much of the race. Doing so ensures that they have sufficient energy at the end for a last-minute sprint (DRH) to the finish line, thereby maximizing their chances of clocking a good time.

QUICK QUIZ G

  1. On a (VD/VI) schedule, reinforcement is contingent upon responding continuously for a varying period of time; on an (FI/FD) schedule, reinforcement is contingent upon the first response after a fixed period of time.
  2. As Tessa sits quietly in the doctor’s office, her mother occasionally gives her a hug as a reward. As a result, Tessa is more likely to sit quietly on future visits to the doctor. This is an example of a(n) ________________ ________________ schedule of reinforcement.
  3. In practicing the slow-motion exercise known as tai chi, Tung noticed that the more slowly he moved, the more thoroughly his muscles relaxed. This is an example of d________________ reinforcement of ________________ ________________ behavior (abbreviated ________________).
  4. In a video game, the faster you destroy all the targets, the more bonus points you obtain. This is an example of ________________ reinforcement of ________________ ________________behavior (abbreviated ________________).
  5. Frank discovers that he feels better if he jogs at a nice, even rhythm that is neither too fast nor too slow. This is an example of ________________ reinforcement of ________________ behavior (abbreviated ________________).

Noncontingent Schedules On a noncontingent schedule of reinforcement, the reinforcer is delivered independently of any response. In other words, a response is not required for the reinforcer to be obtained. Such schedules are also called response-independent schedules. There are two types of noncontingent schedules: fixed time and variable time.

On a fixed time (FT) schedule, the reinforcer is delivered following a fixed, predictable period of time, regardless of the organism’s behavior. For example, on a fixed time 30-second (FT 30-sec) schedule, a pigeon receives access to food every 30 seconds regardless of its behavior. Likewise, many people receive Christmas gifts each year, independently of whether they have been naughty or nice—an FT 1-year schedule. FT schedules therefore involve the delivery of a “free” reinforcer following a predictable period of time.

On a variable time (VT) schedule, the reinforcer is delivered following a varying, unpredictable period of time, regardless of the organism’s behavior. For example, on a variable time 30-second (VT 30-sec) schedule, a pigeon receives access to food after an average interval of 30 seconds, with the actual interval on any particular trial ranging from, say, 1 second to 60 seconds. Similarly, you may coincidentally run into an old friend about every 3 months on average (a VT 3-month schedule). VT schedules therefore involve the delivery of a free reinforcer following an unpredictable period of time. (Question: How do FT and VT schedules differ from FI and VI schedules)?

QUICK QUIZ H

  1. On a non-schedule of reinforcement, a response is not required to obtain a reinforcer. Such a schedule is also called a r________________ – i________________ schedule of reinforcement.
  2. Every morning at 7:00 A.M., a robin perches outside Marilyn’s bedroom window and begins singing. Given that Marilyn very much enjoys the robin’s song, this is an example of a ________________ ________________ 24-hour schedule of reinforcement (abbreviated ________________).
  3. For farmers, rainfall is an example of a noncontingent reinforcer that is typically delivered on a ________________ ________________ schedule (abbreviated ________________).

Noncontingent reinforcement may account for some forms of superstitious behavior. In the first investigation of this possibility, Skinner (1948b) presented pigeons with food every 15 seconds (FT 15-sec) regardless of their behavior. Although you might think that such free reinforcers would have little effect on the pigeons’ behavior (other than encouraging them to stay close to the feeder), quite the opposite occurred. Six of the eight pigeons began to display ritualistic patterns of behavior. 

For example, one bird began turning counterclockwise circles, while another repeatedly thrust its head into an upper corner of the chamber. Two other pigeons displayed a swaying pendulum motion of the head and body. Skinner believed these behaviors developed because they had been accidentally reinforced by the coincidental presentation of food.

For example, if a pigeon just happened to turn a counterclockwise circle before food delivery, that behavior would be accidentally reinforced and increase in frequency. This would increase the likelihood of the same behavior occurring the next time food was delivered, which would further strengthen it. The eventual result would be a well-established pattern of turning circles, as though turning circles somehow caused the food to appear.

Some researchers have argued that Skinner’s evidence for superstitious behavior in the pigeon may not be as clear-cut as he believed. They claim that at least some of the ritualistic behaviors he observed may have consisted of innate tendencies, almost like fidgeting behaviors, that are often elicited during a period of waiting (Staddon & Simmelhag, 1971).

These tendencies, which are discussed in Chapter 12, are known as adjunctive behaviors. Nevertheless, other experiments have replicated the effect of noncontingent reinforcement on the development of superstitious behavior. Ono (1987), for example, placed students in a booth that contained three levers and a counter. The students were told that “if you do something, you may get points on the counter” (p. 263). 

They were also told to get as many points as possible. In reality, the points were delivered on either an FT or VT schedule, so the students’ behavior actually had no effect on point delivery. Nevertheless, most students developed at least temporary patterns of superstitious lever pulling; that is, they pulled the lever as though it were effective in producing points. Interestingly, one student started with lever pulling but then coincidentally received a point after simply touching the counter.

This led to a superstitious pattern of climbing on the counter and touching different parts of the apparatus, apparently in the belief that this action produced the points. She then jumped off the apparatus at just the time that she received another point, which led to a superstitious pattern of repeatedly jumping in the air and touching the ceiling! After several minutes of this, she finally quit, apparently as a result of fatigue.

Professional athletes and gamblers are particularly prone to the development of superstitions, some of which may evolve in the manner that Skinner suggests. Under constant threat of losing their position to an eager newcomer, professional athletes are constantly on the lookout for anything that might enhance their performance. As a result, unusual events that precede a fine performance, such as humming a certain tune or wearing an unusual article of clothing, may be quickly identified and then deliberately reproduced in the hopes of reproducing that performance.

Gamblers display even stronger tendencies toward the development of superstitions, probably because the activity in which they are engaged is even more uncertain in its outcome. Bingo players, for example, commonly carry lucky pendants, stuffed animals, or pieces of jewelry to each game, and they are often adamant (almost pathologically so) about obtaining cards that contain certain patterns or are drawn from the top or bottom of the stack. Many of these rituals probably evolved because they were at one time associated with a big win.

Herrnstein (1966) noted that superstitious behaviors can sometimes develop as by-products of contingent reinforcement for some other behavior. For example, a businessman might believe it is important to impress customers with a firm handshake, when in fact it is merely the handshake, and not the firmness of the handshake, that is the critical factor. (Unfortunately, such a superstition could have serious consequences if the businessman then attempts to branch out into the Asian market, where a firm handshake is often regarded as a sign of disrespect.)

Similarly, some managers might come to believe that “pushing the panic button” is an effective way to deal with crises, simply because it is usually followed by a successful outcome. What they fail to realize is that a low-key approach might have been equally if not more effective—and certainly a lot less stressful.

Question: Although Skinner’s (1948b) original demonstration of superstitious behavior involved the use of a fixed time schedule, you might wish to consider whether superstitious behavior in humans is more likely to develop under a fixed or variable time schedule.

To answer this, think about the types of situations in which you are particularly likely to find superstitious behavior in humans. Is it in situations that involve predictable events or unpredictable events? Obviously, it is unpredictable events, such as games of chance, performance in sports, hunting and fishing (“Jana’s lucky lure”), and so forth. In this sense, at least from a human perspective, superstitious behavior can be seen as an attempt to make an unpredictable situation more predictable.

QUICK QUIZ I

  1. When noncontingent reinforcement happens to follow a particular behavior, that behavior may (increase/decrease) in strength. Such behavior is referred to as s________________ behavior.
  2. Herrnstein (1966) noted that superstitious behaviors can sometimes develop as a by-product of (contingent/noncontingent) reinforcement for some other behavior.
  3. As shown by the kinds of situations in which superstitious behaviors develop in humans, such behaviors seem most likely to develop on a(n) (VT/FT) schedule of reinforcement.

What happens if a noncontingent schedule of reinforcement is superimposed on a regular, contingent schedule of reinforcement? What if, for example, a pigeon responding on a VI schedule of food reinforcement also receives extra reinforcers for free?

Will the pigeon’s rate of response on the VI schedule increase or decrease? In fact, the pigeon’s rate of response on the noncontingent schedule will decrease (Rachlin & Baum, 1972). Just as people on welfare might sometimes become less inclined to look for work, the pigeon that receives free reinforcers will work less vigorously for contingent reinforcers. Suggestive evidence of this effect can even be found among professional athletes.

One study, conducted several years ago, found that major league pitchers who had signed long-term contracts showed a significant decline in number of innings pitched relative to pitchers who only signed a 1-year contract (O’Brien, Figlerski, Howard, & Caggiano, 1981). Insofar as a long-term contract or a guaranteed purse (as in boxing) virtually guarantees a hefty salary regardless of performance, these results are consistent with the possibility that athletic performance might sometimes decline when the money earned is no longer contingent on level of performance.

At this point, you might be thinking that noncontingent reinforcement is all bad, given that it leads to superstitious behavior in some situations and to poor performance in others. In fact, noncontingent reinforcement is sometimes quite beneficial. More specifically, it can be an effective means for reducing the frequency of maladaptive behaviors.

For example, children who act out often do so to obtain attention. If, however, they are given a sufficient amount of attention on a noncontingent basis, they will no longer have to act out to obtain it. Noncontingent reinforcement has even been shown to reduce the frequency of self-injurious behavior.

Such behavior, which can consist of head-banging or biting chunks of flesh out of one’s arm, is sometimes displayed by people who suffer from certain types of developmental disorders, and can be notoriously difficult to treat. In some of these cases, the behavior appears to be maintained by the attention it elicits from caretakers. 

Research has shown, however, that if the caretakers provide the individual with plenty of attention on a noncontingent basis, then the frequency of their self-injurious behavior may be greatly reduced (e.g., Hagopian, Fisher, & Legacy, 1994). In a sense, they no longer have to injure themselves to receive attention because they are now receiving lots of attention for free.

The beneficial effects of noncontingent reinforcement can also be seen as providing empirical support for the value of what Carl Rogers (1959), the famous humanistic psychologist, called “unconditional positive regard.” Unconditional positive regard refers to the love, respect, and acceptance that one receives from significant others, regardless of one’s behavior.

Rogers assumed that such regard is a necessary precondition for the development of a healthy personality. From a behavioral perspective, unconditional positive regard can be viewed as a form of noncontingent social reinforcement, which can indeed have beneficial effects. In fact, it seems likely that proper child rearing requires healthy doses of both noncontingent reinforcement, which gives the child a secure base from which to explore the world and take risks, and contingent reinforcement, which helps to shape the child’s behavior in appropriate ways, maximize skill development, and prevent the development of passivity.

Thus, Abraham Maslow (1971), another famous humanistic psychologist, argued that child rearing should be neither too restrictive nor too lenient, which in behavioral terms can be taken to imply that the social reinforcement children receive should be neither excessively contingent nor excessively noncontingent.

QUICK QUIZ J

  1. During the time that a rat is responding for food on a VR 100 schedule, we begin delivering additional food on a VT 60-second schedule. As a result, the rate of response on the VR schedule is likely to (increase/decrease/remain unchanged).
  2. A child who is often hugged during the course of the day, regardless of what he is doing, is in humanistic terms receiving unconditional positive regard. In behavioral terms, he is receiving a form of nonsocial reinforcement. As a result, this child may be (more/less) likely to act out in order to receive attention.

Complex Schedules of Reinforcement

All of the schedules previously described are relatively simple in that there is only one basic requirement. On the other hand, a complex schedule consists of a combination of two or more simple schedules. There are a wide variety of such schedules, three of which are described here. Two other types of complex schedules—multiple schedules and concurrent schedules—are discussed in later chapters.

Conjunctive Schedules A conjunctive schedule is a type of complex schedule in which the requirements of two or more simple schedules must be met before a reinforcer is delivered. For example, on a conjunctive FI 2-minute FR 100 schedule, reinforcement is contingent upon completing 100 lever presses plus at least one lever press following a 2-minute interval. Many of the contingencies that we encounter in everyday life are examples of conjunctive schedules.

The wages you earn on a job are contingent upon working a certain number of hours each week and doing a sufficient amount of work so that you will not be fired. Likewise, Jon’s fiancée might have chosen to marry him because he is kind and humorous and interesting and drives a Porsche. With any one of these components missing, she would have paid little attention to him.

Adjusting Schedules In an adjusting schedule, the response requirement changes as a function of the organism’s performance while responding for the previous reinforcer. For example, on an FR 100 schedule, if the rat completes all 100 responses within a 5-minute interval, we then increase the requirement to 110 responses (FR 110). In other words, because the rat has performed so well, we expect even better performance in the future.

In a similar fashion, when Tara displayed excellent ability in mastering her violin lessons, she and her parents decided to increase the amount she had to learn each week. And when Lily’s high school students performed poorly on their exams, she gradually decreased the amount of material they had to learn each week. (It is, of course, in this manner that standards in school become gradually lowered, often to the detriment of the students.)

The process of shaping involves an adjusting schedule insofar as the criterion for reinforcement is raised depending on the animal’s performance. As soon as the rat has learned to stand near the lever to get food, the criterion is raised to touching the lever, placing a paw on the lever, and so forth. The requirement for reinforcement changes as soon as the rat has successfully met the previous requirement.

QUICK QUIZ K

  1. A(n) ________________ schedule is one that consists of a combination of two or more simple schedules.
  2. In a(n) ________________ _ schedule, the response requirement changes as a function of the organism’s performance while responding for the previous reinforcer. In a(n) ________________ schedule, the requirements of two or more simple schedules must be met before the reinforcer is delivered.
  3. To the extent that a gymnast is trying to improve his performance, he is likely on a(n) ________________ schedule of reinforcement; to the extent that his performance is judged according to both the form and quickness of his moves, he is on a(n) ________________ schedule.

Chained Schedules A chained schedule consists of a sequence of two or more simple schedules, each of which has its own SD and the last of which results in a terminal reinforcer. In other words, the person or animal must work through a series of component schedules to obtain the sought-after reinforcer. A chained schedule differs from a conjunctive schedule in that the two component schedules must be completed in a particular order, which is not required in a conjunctive schedule.

As an example of a chained schedule, a pigeon in a standard operant conditioning chamber is presented with a VR 20 schedule on a green key, followed by an FI 10-sec schedule on a red key, which then leads to the terminal reinforcer of food.

Thus, an average of 20 responses on the green key will result in a change in key color to red, following which the first response on the red key after a 10-second interval will be reinforced by food. The food is the terminal reinforcer that supports the entire chain.

This chain can be diagrammed as follows:

Note that the presentation of the red key is both a secondary reinforcer for completing the preceding VR 20 schedule and an SD for responding on the subsequent FI 10-sec schedule. Note, too, that this is an example of a two-link chain, with the VR 20 schedule constituting the first, or initial, link and the FI 10-sec schedule constituting the second, or terminal, link. By adding yet another schedule to the start of the chain, we can create a three-link chain, for example:

In this case, both the green and red keys function as secondary reinforcers that help maintain behavior throughout the chain.

QUICK QUIZ L

  1. A chained schedule consists of a sequence of two or more simple schedules, each of which has its own ________________ and the last of which results in a t________________ r________________.
  2. Within a chain, completion of each of the early links ends in a (primary/secondary) reinforcer, which also functions as the ________________ for the next link of the chain.

Once pigeons learn which schedule is associated with which key, they generally show the appropriate response patterns for those schedules. In the preceding example, this would be a moderate, steady rate of response on the white key, a high rate of response on the green key, and a scalloped pattern of responding on the red key.

Nevertheless, responding tends to be somewhat weaker in the earlier links of a chain than in the later links. This can be seen most clearly when each link consists of the same schedule. For example, Kelleher and Fry (1962) presented pigeons with a three-link chained schedule with each link consisting of an FI 60-sec schedule:

The pigeons displayed very long pauses and a slow rate of response on the white key compared to the other two keys. The greatest amount of responding occurred on the red key.

Why would the earlier links of the chain be associated with weaker responding? One way of looking at it is that in the later links, the terminal reinforcer is more immediate and hence more influential, while in the early links, the terminal reinforcer is more distant and hence less influential (remember that delayed reinforcement is less effective than immediate reinforcement).

Another way of looking at it is that the secondary reinforcers supporting behavior in the early links are less directly associated with food and are therefore relatively weak (e.g., the green key is associated with food only indirectly through its association with the red key). From this perspective, a chained schedule can be seen as the operant equivalent of higher order classical conditioning—in which, for example, a tone (CS1) associated with food (US) elicits less salivation than the food does, and a light (CS2) associated with the tone elicits less salivation than the tone does.

Similarly, in the example of the chained schedule, the red key associated with the food is a less powerful reinforcer than the food, and the green key associated with the red key is a less powerful reinforcer than the red key. (If you find that you can no longer remember the concept of higher-order classical conditioning, you should go back and review it.) Assignment: Schedules of Reinforcement

The difference in response strength between the early and later links in a chain is representative of a more general behavioral principle known as the goal gradient effect. The goal gradient effect is an increase in the strength and/or efficiency of responding as one draws near to the goal. For example, rats running through a maze to obtain food tend to run faster and make fewer wrong turns as they near the goal box (Hull, 1932).

Similarly, a student writing an essay is likely to take shorter breaks and work more intensely as she nears the end. Dolphin trainers are well aware of the goal gradient effect. Dolphins who are trained to perform long chains of behaviors have a tendency to drift toward “sloppy” performance during the early parts of the chain, and trainers have to be vigilant to ensure that the dolphin’s behavior is not reinforced when this occurs (Pryor, 1975). (Perhaps the most profound example of a goal gradient, however, is that shown by people who desperately need to urinate and become speed demons as they near the washroom.)

An efficient way to establish responding on a chained schedule is to train the final link first and the initial link last, a process known as backward chaining. Using the pigeon example, the pigeon would first be trained to respond on the red key to obtain food. This will establish the red key as a secondary reinforcer through its association with food. The presentation of the red key can then be used to reinforce responding on the green key. Once this is established, the presentation of the green key can be used to reinforce responding on the white key.

In these examples, each link in the chain required the same type of behavior, namely, key pecking. It is also possible to create behavior chains in which each link consists of a different behavior. For example, a rat might have to climb over a barrier and then run through a tunnel to obtain food. This can be diagrammed as follows:

Note that the sight of the tunnel is both a secondary reinforcer for climbing over the barrier and a discriminative stimulus for then running through the tunnel.

As with the previous examples, backward chaining would be the best way to train this sequence of behaviors. Thus, the rat would first be trained to run through the tunnel for food. Once this is established, it would be taught to climb over the barrier to get to the tunnel, with the sight of the tunnel acting as a secondary reinforcer. In this manner, very long chains of behavior can be established.

In one reported example, a rat was trained to go up a ladder, cross a platform, climb a rope, cross a bridge, get into a little elevator box, release the pulley holding the box, lower the box “paw over paw” to the floor, and then press a button to obtain the food (Pryor, 1975). Of course, each of these behaviors also had to be shaped (through reinforcement of successive approximations to the target behavior). Shaping and chaining are thus the basic means by which circus and marine animals are trained to perform some remarkable feats (see Figure 7.2).

FIGURE 7.2

Through shaping and chaining, animals can be taught to display some surprising behaviors.

Most human endeavors involve response chains, some of which are very long. The act of reading this chapter, for example, consists of reading section after section, until the terminal reinforcer of completing the entire chapter has been attained. Completing each section serves as both a secondary reinforcer for having read that section as well as an SD for reading the next section.

Reading the chapter is in turn part of a much larger chain of behaviors that includes attending lectures, taking notes, and studying, the terminal reinforcer for which is passing the course. Fortunately, backward chaining is not required for the development of such chains, because language enables us to describe to one another the required sequence of behaviors (such as by providing a course syllabus). In other words, for humans, response chains are often established through instructions.

Unfortunately, in the case of very long chains, such as completing a course, the terminal reinforcer is often extremely distant, with the result that behavior is easily disrupted during the early part of the chain (remember the goal gradient principle). This is yet another reason why it is much easier to be a diligent student the night before the midterm than during the first week of the semester. Can anything be done to alleviate this problem? One possibility is to make the completion of each link in the chain more salient (i.e., more noticeable), thereby enhancing its value as a secondary reinforcer.

Novelists, for example, need to write hundreds, or even thousands, of pages before the terminal reinforcer of a completed book is attained. To keep themselves on track, some novelists keep detailed records of their progress, such as charting the number of words written each day as well as the exact dates on which chapters were started and completed (Wallace & Pear, 1977).

These records outline their achievements, thereby providing a much-needed source of secondary reinforcement throughout the process. Similarly, students sometimes keep detailed records of the number of hours studied or pages read. They might also compile a “to-do” list of assignments and then cross off each item as it is completed. Crossing off an item provides a clear record that a task has been accomplished and also functions as a secondary reinforcer that helps motivate us (Lakein, 1973). In fact, it can be so reinforcing that some people will actually add a task to a to-do list after they have completed it, simply to have the pleasure of crossing it off!