Ghostbusters and Behavior Reinforcement

Ready for some Pop Psych?

gbvenk.jpg

Let’s take a look at Reinforcement and Ghostbusters, and by Ghostbusters, I mean the 1984 film written by Dan Aykroyd and Harold Ramis. In the start of the film we are introduced to the character Dr. Peter Venkman (played by Bill Murray) in the process of a humorous experiment with two subjects in which he says the phrase: “I’m studying the effect of negative reinforcement on ESP ability.” This is a scene shown in more than a few Psych 101 courses, where it is lambasted for scientific inaccuracy and ethical violation, but there is actually a great deal we could learn about foundational concepts from it.

In the starting scene, Dr. Venkman holds up cards to both individuals, and they are asked to guess what the symbol hidden on the other side might be. When the male subject responds incorrectly, he is given a mild electric shock, and becomes more and more irritated and averse to the experiment. The other female subject, named Jennifer, is never shown the cards true symbols. She giggles, laughs, and gives her best guess and is reported to be correct by Dr. Venkman five times in a row (even though she is not), thus avoiding the shock, and given copious attention and praise for it. The key term here we are going to look at is called Negative Reinforcement, and in the context of the electric shock, is used incorrectly. However, the audience is clearly aware of Dr. Venkman’s true aims, and deception to the subjects, which is where the humor comes in. [1,2]

So what does negative reinforcement mean? A common misconception is that “negative” means painful or averse, when in fact the term relates more to the removal of a stimulus. Let’s compare it to positive reinforcement. [1,2]

Positive Reinforcement: Adding a stimulus, which increases the probability that the behavior it follows will increase in the future. Think of this like a reward. You work a full shift, and you get a paycheck. You’re more likely to work a full shift again. [1,2]

Negative Reinforcement: This is the removal of a stimulus, which increases the probability that the behavior it follows will increase in the future. Think of this like avoidance. If you ask for no onions on your burger, and you’re given a burger without onions, you have avoided the aversive stimulus. You’re more likely to ask for no onions again.  [1,2]

Both positive and negative reinforcement aim to increase the behavior that they follow. Reinforcement is an effect that strengthens behavior. Think of it this way: “Positive” means add a stimulus or stimuli. “Negative” means subtract or remove a stimulus or stimuli. [1,2]

gbjennifer

Returning to our example, Dr. Venkman is not interested in demonstrating the effect of negative reinforcement on ESP. He is not accurately tracking any real variables associated with “ESP”, but he is using Negative Reinforcement to some degree. [2]

Remember Jennifer? What behavior might she be exhibiting to avoid the electric shock? Clearly, the target behavior is not guessing the symbol correctly.  Instead, Jennifer is engaging socially with Dr. Venkman, she is giggling, guessing confidently, displaying attention to him, and responding eagerly to his comments and expressions. She avoids electric shock in each of those conditions, so it could be said that her attention is the true target of the experiment. Negative Reinforcement is, in a sense, in effect for the shocks for Jennifer, but so is Positive Reinforcement; her attention, giggling, and eager guesses are reinforced by Dr. Venkman’s added encouragement to continue the experiment. [1,2]

Let’s circle back to his other subject, the male who is being shocked repeatedly following incorrect guesses. Dr. Venkman gives him false encouragement by telling him “you have only 75 more to go!”, following the complaints and visible increase in irritation. Here, Dr. Venkman is using what is called Positive Punishment, which is the addition of a stimulus, the electric shock, following the incorrect guess. It could be said from this limited experiment, that using Positive Punishment was the actual independent variable controlled by Dr.Venkman, and that it’s effect was used to fluster the male subject into leaving. The subject’s attention and responses were punished, leading to a decrease in that behavior and the male subject leaving the experiment early. [1,2]

Ethically, very dubious experiment, but its comedic effects do demonstrate some actual psychological (and behavioral) phenomena.

 

Questions? Comments? Leave them below.

 

References:

  1. Cooper, J. O., Heron, T. E., & Heward, W. L. (1987). Applied behavior analysis. Columbus: Merrill Pub. Co.
  2. Murray B., Aykroyd, D., Weaver S., Ramis H., Moranis R., Columbia Pictures Industries (Film). (1984). Ghostbusters. Burbank, CA. RCA/Columbia Pictures Home Video.

Photo Credits: http://ghostbusters.wikia.com

Behavioral Extinction and Extinction Bursts

There a few interesting phenomena we study in behavioral psychology when it comes to the reduction of maladaptive (“bad”) behaviors by their consequences. The process by which a behavior is reduced or eliminated by removing the factors that maintain or reinforce it, is called extinction. That is the end goal. Sometimes it works quickly, and other times it doesn’t. The challenging part is figuring out why.

Where it all starts is in a framework called functionalism. In this framework, we see certain types of behaviors as “operants”; meaning that they operate on their environment in order to accomplish something. They are a response with a purpose. What functionalism does is to take these behaviors, in whatever form they are in, and use the context of the situation or repeated situations, to hypothesize why they may be exhibited by the person. In other words, to find the function of the behavior.

c1

Extinction

The process of extinction is necessitated by knowing the function of the behavior. Extinction works by removing that reinforcement- those maintaining factors of the maladaptive behavior; the thing it is functioning to get, in order to make it useless. Behaviors that do not achieve their function, decrease, and are replaced by more adaptive or useful learned behaviors. This is a basic concept in behavioral science. It’s useful to note that the function of a behavior might not always be apparent, or easy to discern. Sometimes it takes several instances of the same behavior in similar situations to see what the function of that behavior is.

Let’s preface examples with the 4 most common functions of learned (operant) behavior: Escape, Access (Items/Activities), Attention, and Sensory. Most behaviors are governed by these either individually, or in combination. “Control” over specifics of these functions are also theorized to be included in the exact conditions that are preferable to the individual.

c2

Now for some examples:

A child in a grocery store wails and grabs for candy bars on the shelves every time they go to the store.

What’s the maladaptive behavior? Wailing and grabbing.

What might be the hypothesized function of that behavior? Access (to the candybars).

It might be that somewhere along the line, that behavior was reinforced by that candybar being given to quiet the crying, or even that candy bars are a common reinforcer to crying behaviors. The history is useful to know, but not the most important feature in decreasing this behavior; the function is.

What we could do to stop the crying and grabbing is to give the candy bar. This is not extinction. Extinction is when a behavior is weakened and is unlikely to happen again. Giving the candy bar is actually going to satisfy the behavior and reinforce it (strengthen it), by what’s called reinforcement and satiation. It’s satisfying it, making that function stronger. The crying stops, this time, but next time it will be back because it was useful.

So what would an extinction procedure looks like? That would be denying or depriving the function. The crying and wailing would NOT receive the candy bar. Not this time, or every time after. As the crying and grabbing would no longer achieve the candy bar goal, it would decrease, giving the opportunity to teach a replacement behavior that could receive a candy bar (asking nicely, etc).

Here’s another.

An adolescent does a great job when it comes to completing their Math, English, and History homework, but every Tuesday when they’re given Music homework on their clarinet, they tantrum and throw their bookbag.

What is the maladaptive behavior? The tantrum and throwing of their book bag.

What might be the hypothesized function? Escaping the Music homework is most likely. (Remember: Don’t jump to the “Attention” guess too quickly unless you see a beneficial social effect from other people in the situation).

It might be, that when the adolescent screams, yells, and throws their bag, they get a stern talking to, but never actually have to do that music homework afterwards. (This is also a situation where Time-Outs often fail). The screaming stops, the throwing stops, but this is not extinction. Again, it’s reinforcement and satiation. The music homework never gets done, so it’s been effectively escaped. The function of escape is satisfied. Next Tuesday, you would probably see this behavior occur again when the new fresh homework is assigned.

So what would an extinction procedure looks like? Again, denying or depriving the function. The screaming and throwing of the bag would not receive that out from the homework. Guardians or parents might continue to present the homework option, or a variation including help, but that total escape would not occur. In the following situations, those screaming and throwing behaviors would again be met with a consistent presentation of the homework demand (modified or otherwise), and would decrease the screaming and throwing because they do not work, or achieve their function. Here, skills such as taking a small break, breaking the homework down into manageable steps, or even requesting help would be feasible options for replacement behavior.

c3

It just made it worse!…Not really. That is an Extinction Burst.

In an ideal world, that pattern would happen every single time. This is for the most part true, but there is a phenomenon called an Extinction Burst, which throws a wrench into the extinction process, as a treatment. An Extinction Burst is related very closely to the function of the behavior. When the function of the behavior is not achieved, sometimes the behavior increases in intensity and frequency in order to regain that pattern the person was used to. The behaviors actually become worse than before.

This is where teachers, guardians, parents, and even behavioral professionals, want to give up. Why try something that is making it worse? As an old adage goes, sometimes things get worse before they get better. An Extinction Burst is actually a sign of functional weakening. It is a last push of that function’s expression through that behavior. The last vestige of that behavioral pattern being the “fittest” of its repertoire to succeed. If the extinction continues, the burst ends, and that is when a replacement behavior is most likely to be able to be taught.

So what would an extinction burst look like in our previous scenarios?

In scenario #1, with the child and the candy bar. Parents/Guardians might refuse the candy bar and then see the Extinction burst. Grabbing becomes more aggressive, screaming becomes wailing, gasping, sobbing, and lasts longer.

In scenario #2, the adolescent with the music homework might scream louder, use harsher language, throw property harder, and exhibit longer durations or frequency of the destructive behaviors.

Giving in to the behaviors here, satisfying their function, would be easy for most guardians/parents. Unfortunately, it is exactly the purpose of those behaviors and the burst. Extinction Bursts are an effective adaptation. Maybe not prosocial, or useful in the longer term to the individual, but the increase in intensity pays off, from an evolutionary or phylogenic perspective.

The Extinction Burst would end if it is deprived and replaced, and very quickly if there is a replacement behavior that is effective to a degree that would be useful across the lifetime and society. Behaviors that have been useful for longer periods take longer to extinguish, and often have their extinction bursts. It is not an easy process for entrenched maladaptive behaviors with longer histories of success with their function. Certain behaviors (“Please!”/ “Thank You!”, “I need a break from this”), may have less stark effects than their maladaptive counterparts at first glance, but the fact they are useful across almost all situations in the person’s life makes them more likely to be reinforced by consequences in more settings and across a longer span of time.

Comments? Questions? Write them below!

References:

  1. Cooper, J. O., Heron, T. E., & Heward, W. L. (1987). Applied behavior analysis. Columbus: Merrill Pub. Co.
  2. Lerman, D. C., & Iwata, B. A. (1995). Prevalence of the extinction burst and its attenuation during treatment. Journal of Applied Behavior Analysis, 28 (1), 93-94.
  3. Ducharme, JM, Van Houten R. Operant extinction in the treatment of severe maladaptive behavior: adapting research to practice. Behav Modif. 1994 Apr;18(2):139–170

Photo Credits: http://www.pixabay.com (users: martakoton, AxxLC, geralt).

Social Data: Getting the Most of Your Polls, Questionnaires, and Surveys

Economical stock market graph

Self Report Studies:

Self-report studies; you see them all the time online: surveys, polls, and questionnaires. You even see them on your receipts after you order a coffee. Your opinion counts. People want your social data. Some of these are used for marketing, and others are just done for the fun of it.

The reason they are called Self-Report Studies is that they rely on that group to give their own interpretation or information on the subject. The information that the group provides (or self-reports) is called data. Data, especially social data, is important to a lot of people because it gives a voice to a target audience. Many social media networks (Facebook, Twitter, etc) now give their users the ability to do their own self-report studies with only a few clicks of a button. Here are some tips to make sure yours are made with a scientific understanding of the principles!

Sample!

A sample is the selection of a population that you are using for your study (poll, questionnaire, etc). These are the respondents to your self-report study. They are the ones providing your data. Since they make up the entirety of the feedback you will be receiving in your study, you probably want to get the most out of your sample, right?

There are two main ways you can do this. Scope of your study, and size/diversity of your sample.

Scope: The scope of your study, or poll, is the breadth of information you are looking for. It is the net you are casting to catch the information on your topic. [1,2]

You can use a narrow scope: using a topic that is relevant only to the select population you are targeting “Hello, Members of the Elephant Watching Club! Which is your favorite type of elephant?”.

Or, you can use a broad scope: using a topic that has broader relevance on a larger topic: “Hello, Members of the Elephant Watching Club! What do you think about Politician A?”

You might have noticed something in those examples. The survey is only able to track the Members of the Elephant Watching Club. The Elephant Watching Club is the extent of their sample. If the person wanted to get an answer that applied specifically to members within that club, then they would be fine with either scope; just so long as they did not interpret that as an example of a broader population. This leads us to the next one: size/diversity. [1,2]

Sample Size/Diversity:  This refers to the size of your sample, and the diversity of people/opinions within it. If you want to, say, get a representative sample for the United States of America, would you only sample the Elephant Club? Probably not! They most likely do not have the diversity or size to be of use. You would want a sample that is representative, or represents, the diversity in opinion of the target audience you are inferring from.

This may sound difficult for polling. How would you do it? Many researchers use what is called a “Random Sample”, which is a sampling method that gives every member of a population being studied an equal chance of being selected for that sample. It gives broader reach, as well as less hand-picking by the researcher which could lead to bias. If this is something that your current self-report study media does not do, try and adjust your topics to account for Scope, and Sample Size/Diversity! [1,2]

 

Tip! Target your self-report study to fit closely to your sample! Know the population you are studying and gear what you are looking for to fit within that group.

 Desk office business financial accounting calculate, Graph analy

Types of Questions:

There are two types of questions used in self-report studies: Open Questions, and Closed Questions.

Closed questions are questions which provide a limited choice, especially if the answer is taken from a predetermined list. This provides what is called quantitative (numerical) data, but do not allow the participant to give in-depth insights. They are “closed” because they give the respondents a pre-selected set of options to choose from. Polls often used Closed Questions for numerical data. [1,2]

  • Examples: “Pick your favorite ice cream from the following list: Vanilla, Chocolate, Strawberry”. or   “Do you like apples? Yes or No”.

Open questions are those questions which invite the respondent to provide answers in their own words and provide what is called qualitative data.  These questions give you more in-depth answering in the respondent’s own words, but do not allow you to quantify them as easily and compare them to others. They are “open” because the answer may be anything the respondent writes down or replies with. Questionnaires and Surveys include Open Questions for respondent-detailed replies. [1,2]

  • Examples: “Tell us what you think about our service?” or  “What about the apple grove did you like best?”

 

Social Media Example: A Poll would be Closed Question Data. The replies written by users would be Open Question Data.

Each type of data has its own benefits and drawbacks. You would want Closed Questions to provide you data that you could numerically analyze quickly. Everyone responding has the same answers, so it is like comparing apples to apples (so many apple examples…). But, if you wanted a more nuanced answer for the sake of feedback that did not have the same comparability, you could use Open Data to get more detail from your sample. [1,2]

 

Tip! Use the data that fits the type of output you want most. Want descriptive feedback (which you do not need to represent in a graph)? Use Open Questions. Want to make a graph and get numerical data? Use Closed Questions.

 

That’s Mean! (Median, and Mode):

Now let’s talk about some data analysis we can use for quantitative (numerical) data. You might get many responses from your sample that you’ve tailored both the scope, and sample size/diversity for maximum accuracy! What now? Now that you have the data, it’s time to interpret it. Sometimes the media or software you are using would do this for you. If not, take notice of these three terms: Mean, Median, and Mode.  These are what are called “Measures of Central Tendency”, and are used in statistics. If you want to know what most people in your sample are responding, while avoiding fringes; these might be useful to you. [1,2]

Mean is the average of the group of scores you get back. All numbers/responses being equal, this is taking all of them and finding the average response. You do this by adding up all the scores, and then dividing by the sum of the scores. [1,2] It looks like this:

  • Scores: 1, 3, 5, 3, 8. Mean (Average)= 1+3+5+3+8, then dividing by 5 (the number of scores) to get 4.

Median is taking the middle value or score when the responses are arranged from lowest to highest.  [1,2] This gets you a representation of the “middle guy” in the group, and looks something like this:

  • Scores: 1, 3, 3, 5, 8. Median= 3.

Mode is the score that occurs the most within your responses. When you want to see which exact response was chosen over the others, you can look at mode.  [1,2] It looks like this:

  • Scores, 1, 3, 5, 3, 8. Mode= 3. 3 was chosen twice.

 

Tip! Use the measure that gives you the most out of your self-report study! Mean (Average) is the most common you will see and is well liked for easy data output. Mode is when you are curious about the dead center respondents in your sample. Median is what you want if you are curious about the popularity of a specific answer being chosen.

b3rmsmqi4qk-kate-serbin

Oh no! Response Bias:

You’re doing great! You have your sample, you have your scope and representative size/diversity, and now you even have your quantitative measures of central tendency! What could go wrong?

Well, sometimes the people responding. Their bias, or factors that influence how they pick selections in a self-report study, can give us skewed or inaccurate results. Sometimes we are able to adjust our self-report study ahead of time (by wording questions a certain way) to mitigate this, and other times it is simply a part of it. Keep in mind that when ever someone is responding to a survey or poll, it is their interpretation that makes up the data; it is not a direct observation of reality. [1,2] Here are some types of biases that you should be aware of:

Self-Serving Bias: This is when successes are attributed to internal factors (themselves) and failures are attributed to external factors (others). [1,2]

  • Example of a question susceptible to Self-Serving Bias: “Do you feel as though you have been passed over for a job for someone less qualified than you?”

Acquiescence Bias:  This is when respondents say “yes” based not on the question, but rather on the favorability of that response (even though it may be anonymous) to the studier.  [1,2]

  • Example of a question susceptible to Acquiescence Bias: “Look at this pic! Am I pretty today? Yes or No!”

Extreme Responding Bias: This is where respondents prefer to pick the most extreme responses possible from a selection. (ie. Something is “literally the best/worst!”). [1,2]

  • Example of a question susceptible to Extreme Responding Bias: “On a scale from 1 to 10, how good was that episode of TVSHOW?” 10! 10! 10! 1! 10!

Social Desirability Bias: In this one, people respond with the most socially appropriate (or inappropriate) answers that conform to the expected desirability of the group or studier. [1,2]

  • Examples of questions susceptible to Social Desirability Bias:  “Do you give to charity? Yes or No”, “Do you ever have rebellious instincts?? Yes or No!”

Do you see how these biases may affect the interpretation of data? Keep them in mind!

 

Tip! Know your audience, and know your questions. Even one favorable (or unfavorable) word in a question could get your respondents to reply according to these biases. Think about what type of biases may be expressed when answering your questions.

t5bva-q_m_y-luis-llerena.jpg

Get it Valid and Reliable:

There is one more important thing when you are studying a topic in social science: Validity and Reliability. These are factors that the person studying and presenting the self-report study should build in as best as possible before sending out their self-report materials to the world. These factors are what you use to make sure that you are studying something real, and that you are studying it accurately! [1,2]

Validity is the ability for a test/study to measure what it is intended to measure.  An example of this might be, if you are trying to study something like people’s opinions on a specific topic, does your question cover it, and is that question worded to target the topic specifically? [1,2]

  • Example: We are studying whether people enjoy the taste of chocolate. We ask “Do you enjoy the taste of chocolate?” 76% say yes! So long as everything is defined and specific, we could call this valid. 76% of respondents enjoy chocolate.
  • Non-Example: Whether people enjoy the taste of chocolate. We ask “Do you like sweet foods?” 84% say yes!, and in our study we conclude that that 84% of people enjoy chocolate. Wait a minute. Was that our question? Was our question tailored to fit the validity of the study? We used Sweet in our question, but we concluded on the factor of Chocolate. This is not valid.

Reliability is the ability of our test to yield (nearly) the same result each time we test with it. If we are able to test a sample with these questions, and provide an alternative test (on the same topic), we would get similar responses both time. The reason we need this is to be sure that it is not a fault or mistake in the test that is giving an inaccurate conclusion. Sometimes biased-wording, text errors, or jargon, can lead to responses being skewed or erratic. If 2 tests, or the same test twice, can get stead and similar responses from the same population, then we know that variability in responses is based on the respondents, and not our questions. [1,2]

  • Example: If you run the Chocolate Preference Test twice, and the first set of responses equal 80% while the next equals 81%; this is as close to reliable as you might be able to expect.
  • Non-Example: If you run the Chocolate Preference Test twice, and the first set of responses equal 17%, while the next equals 54%; there is something wrong. Assuming this is the same sample or even population, you might want to look at your test as a factor which influenced results incorrectly.

Both of these methods assure that you, the designer of the study, are not including factors that could effect the results you get. You want your results to match the respondents, not artifacts (unrelated data) embedded in your questions.

 

Tip! Test and re-test. If you have an audience, rephrasing questions on the same topic and presenting them again may get you a better picture when you keep validity and reliability in mind.

Questions? Comments?

 

References:

  1. Wood, S. E., Wood, E. R., & Wood, E. R. (1996). The world of psychology. Boston: Allyn and Bacon.
  2. Cooper, J. O., Heron, T. E., & Heward, W. L. (1987). Applied behavior analysis. Columbus: Merrill Pub. Co.

Photo Credits: https://stock.adobe.com, http://www.unsplash.com (Luis Llerena, Kate Serbin)

Psych Terms: Stimulus Pairing

What is Stimulus Pairing? What why is it important? I would describe it this way. If you wanted to understand the foundation of how we learn (or any creature with a complex nervous system or brain), then stimulus pairing would be one of the first building blocks of the process you would come across. It explains the process of our reactions and future interpretation of stimuli in our environment. To understand it; let’s start with the original researcher Ivan Pavlov, and his discovery: classical conditioning.

5ylv-khwm9s-anita-peeples

Classical Conditioning

Classical conditioning (or respondent conditioning) is a learning procedure in which a biologically potent stimulus (like water, food) is paired with a previously neutral stimulus (in Pavlov’s case; a bell). It also can refer to the learning process that results from this kind of pairing, through which the neutral stimulus (the bell) comes to elicit a response/reflex (like salivation) that is usually like the one elicited by the potent stimulus. [1,2,3]

These were first studied in detail by Ivan Pavlov through experiments with dogs. Together with operant conditioning, classical conditioning became the foundation of behavioral psychology (behaviorism).

Review of Pavlov’s experiment:

Pavlov noticed that his dogs began to salivate in the presence of the researcher who normally fed them, rather than simply salivating in the presence of food. Pavlov called these “anticipatory salivations”, and wanted to explore the phenomenon more. To do so, Pavlov presented a stimulus (the sound of a bell ringing) and then gave the dog food. After a few repeated trials, the dogs started to salivate in response to the stimulus. Pavlov concluded that if a stimulus (the ringing bell) in the dog’s surroundings was present when the dog was given food then that stimulus could become associated with food and cause salivation on its own. This is the gist of stimulus pairing. [1,3]

dltpk6n31fc-igor-ovsyannykov

Stimuli (and Responses) in Stimulus Pairing

Pavlov called the stimulus being paired the conditioned stimulus (CS) because its effects depend on its pairing with food. He called the food the unconditioned stimulus (US) because its effects did not depend on previous experience. This same terminology is also used for responses! They can both be conditioned (trained by stimulus pairing), or unconditioned (“natural”, “reflex”). [1,3]

Example Time!

Unconditioned Stimuli- Food, water, sleep, light, temperature, et cetera. These are all considered “unconditioned” because we do not need to be taught to react to them. Many of these are either biological necessity or things that we respond to reflexively.

Conditioned Stimuli- These are things that are “learned” or take on new meaning by being paired with one of those unconditioned reinforcers. Let’s use a food example again. You hear a jingle, or advertisement, for your favorite restaurant. It may take on the effects of that previous experience with the food.

Unconditioned Response- Examples of these are more often than not reflexes. Your pupils widening to light. Salivation when in the presence of food. These are untrained and happen regardless of history.

Conditioned Response- These are trained responses. Remember that food example with the conditioned stimuli? The food jingle. A response to that might be salivation, or seeking (appetitive) behavior for that restaurants food. These are commonly seen with phobias or aversions as well. If you get in to a car accident in your red car, your response in the future might be to react with panic or anxiety in the presence of another red car.

 

Make sense? What other scenarios can you think of? What stimuli have you been conditioned to?

 

Comments? Questions? Write them below!

 

References:

  1. Cooper, J. O., Heron, T. E., & Heward, W. L. (1987). Applied behavior analysis. Columbus: Merrill Pub. Co.
  2. L. Brink (2008) Psychology: A Student Friendly Approach
  3. Shettleworth, Sara J.(2010)Cognition, Evolution, and Behavior (2nd edn) Oxford Univ. Press

Photo Credits: http://www.unsplash.com

Behavioral Science In Video Games

In behavioral science we like to look at things that are concrete and observable. Why do people respond to specific scenarios and stimuli in different ways? How do they differ from one another? How can we adapt what we present in ways that either increase or decrease a person’s responding? These are questions we can apply to our area of interest; Video Games, in order to explore what game designers have put in to their medium to get you hooked and keep you hooked. Video Games require the audience to participate in ways that other art mediums do not. It is the direct responses of the consumer that shape and define their progress through the game and a hallmark trait of video games is using rewards as marks of progress that get people to play longer, increase their own skill at the game, and master objectives that the designers put in place. Let’s discuss some of the behavioral principles that may in play with the games you know and love. See if you can identify these concepts in your own experiences with video games.

arcade-1254453
Reinforcement vs. Rewards

In behavioral science, we use the word reinforcement to define a consequence that strengthens a future behavior, when presented with the same setting/stimulus (antecedent). When a reinforcer is presented after a behavior, we expect to see the probability of that behavior to go up the next time the person is placed in that situation. It is the foundation of learning and operant behavior. Operant Behavior is a large piece of this conceptual puzzle; it is behavior that has been shaped to serve a purpose in the environment, which has been reinforced in the past. How does this differ from rewards? In gaming of all types, there are rewards. These are pre-set consequences or prizes that follow the completion of specific objectives laid out for the player. Some prizes/rewards are interesting to a player and keep them engaged with the game, and others do not, leading to disinterest or a falloff in responding (playing). What makes a reinforcer different from a reward, is that reinforcers are dependent on the individual’s future responding. When we say reinforcer, we are saying with a degree of certainty that this “reward” has effected behavior before and is preferred by the individual, because it has been shown to have worked in the past. Let’s look at this scenario:

Player 1 must press the circle button when presented with a box in order to break the box and gain a prize (100 points).

If Player 1 presses the circle button and breaks the box, and gets the 100 points, they have been “rewarded”.

If Player 1 presses the circle button and breaks the box, gets the 100 points, and presses the circle button when presented with more boxes in the future, they have been reinforced.

It could be said that 100 points was enough to reinforce the behavior. This effects future playing behavior by pairing a preferred stimulus (the points) with an operant behavior (pressing the circle button) in the presence of the box (antecedent). This is also called the Three Term Contingency.

If game designers want their players to learn certain skills specific to their game, or keep people playing it, they need to focus on casting the widest net of reinforcers, rather than just rewards. Anything can be a reward, but only when it’s considered a reinforcer, will we see players use those skills to progress again and again.

p1
Schedules of Reinforcement:

In the example above, we have a single situation, with a single reinforcer. Games are made up of varied scenarios, competing choices for the player to take, and sometimes we see two types of reinforcement used at the same time. How does that work? Sometimes a player is presented with an opportunity to complete two objectives at the same time. This brings a level of challenging complexity that most players enjoy more than a simplistic single system of reward, because it raises the stakes in terms of what they can receive. Let’s take a look at some simple schedules of reinforcement below:
Fixed Ratio Reinforcement:
In this schedule of reinforcement, we see a set rate of responding met with a set amount of reward. So if a player beats 1 adversary and receives 200 points, this is called a FR1 (fixed-ratio 1) ratio. If a player needs to beat 2 adversaries to receive 200 points, this is called a FR2 ratio, and so on. The benefit of this style of reinforcement schedule is that it is consistent and a player can depend on it. If they can predict the amount of points/rewards they receive for each action, they can match their responding to the amount of reinforcement which satisfies them.
Variable Ratio Reinforcement:
Some people know this schedule of reinforcement from RNGs (Random Number Generators) that are put in games to provide variability, and also for some people, a very strong system of reinforcement. Gambling also runs on this principle. With variable ratio, there is percentage that the response will be rewarded. Unlike the Fixed Ratio, prediction of the reinforcer does not follow a fixed series. The Player must rely on chance, or repetition of responses (for more opportunities) in order to receive a reward. Sometimes this can come in the form of an increase in magnitude of the reward (an adversary sometimes is worth 100 points, but may also be worth 500), or frequency (some adversaries reward points, others do not). As we may expect, the chance to receive a large reward for a standard amount of effort can be a very reinforcing contingency.

Looking at these two schedules, we can expect that both have their respective fans. Some players prefer predictability and something that can be planned for. A specific amount of successful responding would equal an expected amount of reward, every time (Fixed Ratio). Others, enjoy the variability; sometimes even a standard amount of responding could pay off in a huge reward (Variable Ratio). When we combine two or more simple schedules, we get the complex schedules:

If you give the player the option between a Fixed Ratio and a Variable Ratio, we call this a concurrent schedule of reinforcement. It would look something like this:
If a player walks down path A to fight the goblins, they can expect 100 points for each goblin adversary beaten, but if the player goes down path B to fight the birds, there is a variable chance of getting 800 points for each bird beaten. Both of these options are available and do not necessarily reduce the option of pursuing the other. A player could fight the goblins for a little while, then choose to fight the birds. The options are both available, thus concurrent. You see these schedules of reinforcement common in games that allow for free exploration, or multiple avenues to the same objective.

If we give the player both a Variable Ratio and Fixed Ratio at the same time, we call that a superimposed schedule of reinforcement. It would look something like this:
A player is set in a scenario where they had to face both goblin adversaries and bird adversaries at the same time. Each goblin adversary that they beat would reward them 100 points (Fixed Ratio), and each bird adversary beaten would give a chance of getting 800 points (Variable Ratio). These two schedules are now running at the exact same time, and the player has the opportunity to pursue each simultaneously.

These are just a few examples of the type of reinforcement schedules you may come across in games. There are no real limits to how many schedules of reinforcement may run concurrently or superimposed. You could run multiple fixed intervals at the same time (An orange is worth 100 every time, an apple is worth 200 points every time), multiple variable ratios (An orange is sometimes worth 100 points, an apple is sometimes worth 200 points). The possibilities are limitless. There even exist schedules of reinforcement that rely on intervals of time, rather than responding (every 3 minutes you receive 100 points, or sometimes every 10 minutes you receive 100, regardless of what responding the player is engaged in).

It stands to reason, however, that the more schedules which run at the same time, and the more complicated the contingencies of reinforcement, the greater the risk that the player will not understand what responses or choices are actually being reinforced. This may lead to some misattribution, or superstitious responding (responding that has been reinforced by a contingency that did not actually exist). When reinforcement schedules are too complex or not clear, they can create confusion with the players, and result in loss of responding or interest in the game.

13010692_1258403490840429_1021608008296563206_n
Complications:

Human behavior is not always easily predicted, and even in video games, game designers can create vast systems of intertwined schedules of reinforcement that keep players enthralled for hours, but there may come a point where the expectations of player responding do not match the predictive models. We have to be aware of some of the other factors in behavioral science and research that influence a decrease in responding (playing) or disinterest. Below are just a few of these that we commonly come across in video games.
Punishment: Punishment is a condition where a stimulus is either presented or removed that decreases the probability a behavior would happen in the future. It serves the opposite purpose of reinforcement. It comes in two variations; positive and negative. These terms do not reflect anything “good” or “bad” but rather an addition or subtraction of stimuli which has a marked effect on the decrease of future behavior when given the same (or similar scenarios). In video games, they look something like this:

• Positive Punishment: A player walks in to a hole. That player receives damage. The hole is the presentation of a stimulus, and assuming damage is aversive to this player’s style or goals, they would be much less likely to walk in to it again.

• Negative Punishment: A player buys an overpriced item in an in-game shop. Assuming the player has lost a significant amount of something that was preferred in exchange for something non-preferred, they are not likely to repeat the buying behavior in the future.
S Δ (S-Delta): S-Delta shares a similarity with Punishment in that it does not strengthen or reinforce a behavior or series of responses. An S-Delta is a stimulus that when present, a particular behavior receives no reinforcement. An example of this might be, if a player is used to running down a path to pick up items/points, the hold down the “Run” button to increase their reinforcement. However, if this same behavior was attempted when in the presence of a wall (S-Delta), that behavior of holding the “Run” button would not receive the same reinforcement. Running behavior is not necessarily punished overall, but it is less likely to be used for reinforcement in the presence of the wall.
Ratio Strain: Ratio Strain is a condition where an increase in response is expected, but the reinforcement is not enough to maintain it. An example of this may be, if a player is used to defeating goblins for 100 points, but is then presented with Super Goblins rewarding 100 points which are much more difficult to defeat, the amount of reward is no longer reinforcing enough to maintain the repetition of responding. This can often be solved by raising the amount of reinforcement to match the effort.
Satiation: Satiation is a common modifying condition for human behavior. There comes a point when a specific reinforcer is acquired to the point where it is no longer a reinforcer anymore. An example of this is, if a player is satisfied with having 10,000 points, and achieves 10,000 points, any future accumulation of points would not reinforce the behavior to continue. The reward condition would remain, but it would no longer be considered reinforcing. This may often be solved by allowing some time to pass to the point where that satiation condition is no longer present, or changing reinforcers.
Response Effort: It is the amount of effort a person has to put forward to complete a target behavior. This is not a barrier to playing in itself, but could denote a change in difficulty. So if we are reinforcing the behavior of defeating ghosts or eating dots, the amount of effort may be how fast a person has to respond to obstacles, or the amount of fine motor skill necessary to navigate to the objective. If the amount of effort exceeds what a player can respond to, we can say the response effort has been set too high to be reinforced.
The Social Factor

We would be remiss in ignoring one of the strongest forms of reinforcement that may not necessarily be provided in the game, but the product of success or even the pursuit of playing could give us; social reinforcement. Sometimes players enjoy the thrill of competition (competitive multiplayer), others enjoy jolly cooperation (cooperative multiplayer). Many find strong reinforcement in sharing their experiences (streaming), or showing off completed objectives (trophies/completion). Bringing other people in to the experience of interacting with video games is by no means a new prospect, but quantitatively measuring social reinforcement in video games is still very much an avenue of research worth pursuing. Some examples that game designers may be able to follow to collect that data may be; how many times multiplayer aspects are utilized, the duration of multiplayer aspects to their game, viewership in streamed media, and of course, consumer demands for specific social aspects that would be feasible in a game. There may also be examples where developed games rely too much on external social reinforcement without providing sufficient contingencies of their own within the game’s design.

m1
Balancing it all

Video Games are rich examples of how human behavior interacts with digital entertainment, and the concepts above are just the tip of the iceberg. Some games employ one or two of those concepts, others employ complex systems of intentional reinforcement and punishment. With different generations we have seen popular features rise and fall but all seem to follow the basic principles; objectives, responses, and rewards. Reading this, you may have some ideas on some other phenomena that might have an effect on the relation between video game and human. The concepts above is in no way exhaustive, but it’s a topic we may be able to explore a deeper in the future. Leave comments below with your thoughts, theories, and opinions.
References:

  • Fantino, Edmund; Romanowich, Paul. (2007) THE EFFECT OF CONDITIONED REINFORCEMENT RATE ON CHOICE: A REVIEW. Journal of the Experimental Analysis of Behavior
  • Magoon, Michael A; Critchfield, Thomas S. (2008) CONCURRENT SCHEDULES OF POSITIVE AND NEGATIVE REINFORCEMENT: DIFFERENTIAL-IMPACT AND DIFFERENTIAL-OUTCOMES HYPOTHESES. Journal of Applied Behavior Analysis
  • Pietras, Cynthia J; Brandt, Andrew E; Searcy, Gabriel D. (2010) HUMAN RESPONDING ON RANDOM-INTERVAL SCHEDULES OF RESPONSE-COST PUNISHMENT: THE ROLE OF REDUCED REINFORCEMENT DENSITY. Journal of Applied Behavior Analysis
  • Pipkin, Claire St Peter; Vollmer, Timothy R (2007). APPLIED IMPLICATIONS OF REINFORCEMENT HISTORY EFFECTS. Journal of Applied Behavior Analysis.
  • Skinner, B. F. (1953). SCIENCE AND HUMAN BEHAVIOR. New York: Free Press.
  • Skinner, B.F. (1938). THE BEHAVIOR OF ORGANISMS. D. Appleton & Company.

Photo Citations:

  1. “Dark Souls 3” – Ethan Russel
  2. “Mario” -Freeimages.com
  3. “Pacman”- Freeimages.com
  4. “Arcade”-Freeimages.com