# Install packages
# install.packages("tidyverse") # manipulate data
# install.packages("Matrix") # calculate rank
## these are the packages we will use for today
# load packages
library(tidyverse)
library(Matrix) Teaching Materials
POL 211 - Introductory Graduate Methods
Here you can find discussion section materials for POL 211 (Fall 2025) at UC Davis. This content is adapted from material prepared by and shared from Professors Yu-Shiuan (Lily) Huang (National Chengchi University) and Ryan Hubert (London School of Economics) for prior iterations of this course.
Download .qmd for Discussion Section I
Download .qmd for Discussion Section II
Discussion Section III
Note: These materials were adapted from content prepared by Professors Yu-Shiuan (Lily) Huang (National Chengchi University) and Ryan Hubert (London School of Economics).
2. Relationships in Data
When we want to describe the relationship between two sets of data, we can plot the data sets in a scatter plot and look at four characteristics:
- Direction: Are the data points sloping upwards or downwards?
- Form: Do the data points form a straight line or a curved line?
- Strength: Are the data points tightly clustered or spread out?
- Outliers: Are there data points far away from the main body of data?
Today, our focus will be on the first three characteristics. We will delve into covariance, the correlation coefficient, and linear regression. These are three fundamental statistics that allow us to describe the relationships between the variables of interest.
a. Visualizing Relationships – Scatter Plot
For today, we are using Covid19.csv. You can download this csv file from Canvas or from here.
## load packages and import data
library(tidyverse)
cf <- read_csv("POL 211/data/Covid19.csv")Let’s visualize the relationship between the politics of a county (dem16vs) and the share of people who say they always wear a mask (mask_always) in a scatter plot.
dem16vs: Democratic vote share in 2016 presidential election.mask_always: The estimated share of people in a county who would say they always wear a mask in public when they expect to be within six feet of another person.
ggplot(cf, aes(x = dem16vs, y = mask_always)) +
geom_point(color = "black", fill = "white",
stroke = 1, shape = 1) +
## Use the stroke aesthetic to modify the width of the border
## The shape of points can be adjusted by specifying the shape argument
xlab("Democratic Vote Share, 2016 Presidential") +
ylab("% Reporting Always Wearing Mask") +
ylim(0,1) +
xlim(0,1) +
labs(title = "NY Times Covid-19 Survey",
subtitle = "All Counties") +
theme_bw()
The plot above is hard to see since all the dots are on top of each other. So instead of plotting all the datapoints, let’s draw a random sample of 500 of the rows of the dataset and plot them.
set.seed(110) ## this allows us to draw the *same* random sample every time.
samp <- sample(1:nrow(cf), 500)
## This means randomly taking a sample of 500 from the elements of the total rows of cf
glimpse(samp) int [1:500] 2008 2790 2483 336 2403 2921 1659 772 1427 2931 ...samp is the rows we randomly sample from all the rows in cf. For example, the first element in samp is 2008, which means we’re going to draw the 2008th row out of the original dataset.
Now let’s replot the scatter plot again with only 500 datapoints we randomly drew from the whole dataset.
ggplot(cf[samp,], aes(x = dem16vs, y = mask_always)) +
## Specify the random sample within the bracket of cf and remember to put it on the row position!
geom_point(color = "black", fill = "white",
stroke = 1, shape = 1) +
xlab("Democratic Vote Share, 2016 Presidential") +
ylab("% Reporting Always Wearing Mask") +
ylim(0,1) +
xlim(0,1) +
labs(title = "NY Times Covid-19 Survey",
subtitle = "500 Randomly Selected Counties") +
theme_bw()
Be cautious about the axis scaling! Choosing how you scale the x and y axis doesn’t change the distribution or correlation, but it can mislead people!
p1 <- ggplot(cf[samp,], aes(x = dem16vs, y = mask_always)) +
geom_point(color = "black", fill = "white",
stroke = 1, shape = 1) +
xlab("Democratic Vote Share, 2016 Presidential") +
ylab("% Reporting Always Wearing Mask") +
ylim(-10,10) + ## change the scale of y axis
xlim(0,1) +
labs(title = "NY Times Covid-19 Survey",
subtitle = "500 Randomly Selected Counties") +
theme_bw()
p2 <- ggplot(cf[samp,], aes(x = dem16vs, y = mask_always)) +
geom_point(color = "black", fill = "white",
stroke = 1, shape = 1) +
xlab("Democratic Vote Share, 2016 Presidential") +
ylab("% Reporting Always Wearing Mask") +
xlim(-2,2) + ## change the scale of x-axis
ylim(0,1) +
labs(title = "NY Times Covid-19 Survey",
subtitle = "500 Randomly Selected Counties") +
theme_bw()
ggpubr::ggarrange(p1, p2, ncol = 2, nrow = 1)
b. Visualizing Relationships – Crosstab/Contingency Table
A crosstab or contingency table is also a way to observe the relationship between two variables.
Let’s look at the relationship between Covid-19 deaths and political breakdown of counties (whether the county voted for Trump or Clinton in 2016 presidential election). The goal is to replicate the same table as the one in the lecture slide (counties by vote choice and deaths/capita).
First, let’s create some new variables.
Looking at the variable on Covid-19 deaths, some of these counties are bigger, so will have more deaths. To account for this, we have to rescale the deaths data to be per capita.
cf <- cf %>% mutate(deaths.percap = c19deaths/population) cf %>% select(county_name, c19deaths, population, deaths.percap) %>% head()# A tibble: 6 × 4 county_name c19deaths population deaths.percap <chr> <dbl> <dbl> <dbl> 1 Autauga County, Alabama 1563 55869 0.0280 2 Baldwin County, Alabama 1858 223234 0.00832 3 Barbour County, Alabama 331 24686 0.0134 4 Bibb County, Alabama 257 22394 0.0115 5 Blount County, Alabama 228 57826 0.00394 6 Bullock County, Alabama 1025 10101 0.101Generate a dummy variable indicating whether the county voted for Trump
cf <- cf %>% mutate(trump = ifelse(dem16vs < 0.5, "Voted Trump", "Voted Clinton")) cf %>% select(county_name, dem16vs, trump) %>% head() ## check the new var to see if we create it correctly# A tibble: 6 × 3 county_name dem16vs trump <chr> <dbl> <chr> 1 Autauga County, Alabama 0.238 Voted Trump 2 Baldwin County, Alabama 0.194 Voted Trump 3 Barbour County, Alabama 0.465 Voted Trump 4 Bibb County, Alabama 0.212 Voted Trump 5 Blount County, Alabama 0.0843 Voted Trump 6 Bullock County, Alabama 0.749 Voted ClintonGenerate a categorical variable indicating which deaths per capita quartile a county is in.
n25 <- ceiling(nrow(cf) * 0.25) ## The first quartile is in the 778th row n50 <- ceiling(nrow(cf) * 0.50) ## The second quartile is in the 1555th row n75 <- ceiling(nrow(cf) * 0.75) ## The third quartile is in the 2333rd row cf <- cf %>% arrange(deaths.percap) %>% ## Arrange the data based on the order of values in deaths.percap mutate(case_id = row_number()) %>% mutate(deaths.quartile = case_when( case_id >= 1 & case_id <= n25 ~ "1st Quartile Deaths/Capita", case_id > n25 & case_id <= n50 ~ "2nd Quartile Deaths/Capita", case_id > n50 & case_id <= n75 ~ "3rd Quartile Deaths/Capita", case_id > n75 ~ "4th Quartile Deaths/Capita", ))Now, let’s create the crosstab: count the number of counties under different quartile deaths/capita and which candidate they voted for (Trump or Clinton).
crosstab <- cf %>% count(deaths.quartile, trump) %>% spread(trump, n) %>% ## After the count operation, this spreads the trump variable into separate columns, one for each unique value in the trump variable. It places the counts (n) for each trump value into their respective columns. mutate(Total = `Voted Clinton` + `Voted Trump`) ## Generate a column for the row total Total <- c("Total", unname(colSums(crosstab[, 2:4]))) ## Generate a row for the column total crosstab <- crosstab %>% rbind(Total) knitr::kable(crosstab, "simple", align = "llll")deaths.quartile Voted Clinton Voted Trump Total 1st Quartile Deaths/Capita 42 736 778 2nd Quartile Deaths/Capita 60 717 777 3rd Quartile Deaths/Capita 88 690 778 4th Quartile Deaths/Capita 212 565 777 Total 402 2708 3110
A crosstab does not make it easier to see the relationship. Usually, when the two variables you are interested in are cardinal ones (both dem16vs and deaths.percap are continuous variables), plotting a scatter plot is more straightforward to see the relationship. In general, crosstab/contingency table is more useful when the two variables you are interested in are both categorical variables.
{r, message=F, error=F, warning=F, fig.align='center', fig.width=5, fig.height=4} ggplot(cf[samp,], aes(x = dem16vs, y = deaths.percap)) + geom_point(color = "black", fill = "white", stroke = 1, shape = 1) + xlab("Democratic Vote Share, 2016 Presidential") + ylab("Deaths Per Capita") + xlim(0,1) + labs(title = "NY Times Covid-19 Survey", subtitle = "500 Randomly Selected Counties") + theme_bw()
c. Covariance, Correlation Coefficient, Linear Regression
In addition to visualizing data using scatter plots or contingency tables to grasp relationships, how can we employ statistical methods to describe these relationships more formally?
- Covariance
Covariance is a useful measure for describing the direction of the linear association between two quantitative variables.
\[Cov(x, y) = \frac{1}{N-1}\sum_{i=1}^{N}(x_i-\bar{x})(y_i-\bar{y})\] As we can see from the equation, the covariance sums the term \((x_i-\bar{x})(y_i-\bar{y})\) for each data point, where \(\bar{x}\) is the average \(x\) value, and \(\bar{y}\) is the average \(y\) value. The term becomes more positive if both \(x\) and \(y\) are larger than the average values in the data set, and becomes more negative if smaller. As the covariance accounts for every data point in the set, a positive covariance must mean that most, if not all, data points are in sync with respect to \(x\) and \(y\) (small \(y\) when \(x\) is small or large \(y\) when \(x\) is large). Conversely, a negative covariance must mean that most, if not all, data points are out of sync with respect to \(x\) and \(y\) (small \(y\) when \(x\) is large or large \(y\) when \(x\) is small).
Let’s start by using a fake dataset to practice how to compute covariance.
tf <- data.frame(x = c(3, 2, 1, 7, -1),
y = c(8, 10, 13, 2, 8))
knitr::kable(tf, "simple", align = "cc")| x | y |
|---|---|
| 3 | 8 |
| 2 | 10 |
| 1 | 13 |
| 7 | 2 |
| -1 | 8 |
## Covariance by hand
#I've started the necessary code below...
n <- length(tf$x)
mean.x <- mean(tf$x, na.rm = TRUE)
mean.y <- mean(tf$y, na.rm = TRUE)
#How would I write this out?
covariance.tf <- (1/(n-1))*sum((tf$x - mean.x)*(tf$y - mean.y))You can use the canned function cov() to double check your answer.
## Covariance
cov(tf$x, tf$y)[1] -8.85Covariance is a useful measure at describing the direction of the linear association between two quantitative variables, but it has two weaknesses: a larger covariance does not always mean a stronger relationship, and we cannot compare the covariances across different sets of relationships. For example, if we find that the covariance between variables \(x\) and \(y\) is larger than the covariance between \(x\) and \(z\), we can only conclude that both \(y\) and \(z\) are positively associated with \(x\). However, we cannot determine whether the relationship between \(x\) and \(y\) is stronger than the relationship between \(x\) and $z based solely on their covariances.
- Correlation Coefficient
To account for the weakness, we normalize the covariance by the standard deviation of the \(x\) values and \(y\) values, to get the correlation coefficient. The correlation coefficient is a value between -1 and 1, and measures both the direction and the strength of the linear association. One important distinction to note is that correlation does not measure the slope of the relationship — a large correlation only speaks to the strength of the relationship. Some key points on correlation are:
- Correlation measures the direction and strength of the linear association between two quantitative variables.
- Positive and negative indicates direction, large and small indicates the strength.
- Correlation has symmetry: correlation of x and y is the same as correlation of y and x.
- Correlation is unitless and normalized.
\[r_{xy}=\frac{Cov(x, y)}{s_xs_y}\]
Please compute the correlation coefficient between \(x\) and \(y\) using the formula above with the fake dataset
## Correlation by hand
#How would I finish it?
covariance.tf <- (1/(n-1))*sum((tf$x - mean.x)*(tf$y - mean.y))
correlation.tf <- covariance.tf/(sd(tf$x)*sd(tf$y))You can also used the canned function cor() to double check your answers.
## Correlation
cor(tf$x, tf$y)[1] -0.7412154- Linear Regression
Correlation and covariance are quantitative measures of the strength and direction of the relationship between two variables, but they do not account for the slope of the relationship. In other words, we do not know how a change in one variable could impact the other variable. Regression is the technique that fills this void — it allows us to make the best guess at how one variable affects the other variables. The simplest linear regression, which is usually measured by ordinary least square (OLS) method, allows us to fit a “line of best fit” to the scatter plot, and use that line (or model) to describe the relationship between the two variables. The OLS approach aims to fit a line that minimizes squared residuals. The equation for that line is:
\[\hat{y} = a + bx\]
The equations of \(a\) and \(b\) are:
\[b = \frac{r_{xy}s_{y}}{s_{x}}=\frac{Cov(x, y)}{s^2_{x}}\]
\[a = \bar{y}-b\bar{x}\]
Note: A more detailed introduction regarding linear regression will be discussed in POL 212 and 213.
What are the intercept (\(a\)) and slope (\(b\)) between \(x\) and \(y\) in the fake dataset? Please compute in a manual way!
##Calculating linear regression coefficients by hand:
## Slope
beta <- ((1/(n-1))*sum((tf$x - mean.x)*(tf$y - mean.y)))/var(tf$x)
beta <- cov(tf$x, tf$y)/var(tf$x)
## Intercept
alpha <- mean(tf$y) - beta*mean(tf$x)
beta[1] -1.005682 alpha[1] 10.61364You can also use the canned function `lm` to check your answer. reg <- lm(y ~ x, data = tf)
stargazer::stargazer(reg, type = "text", digits = 4)
===============================================
Dependent variable:
---------------------------
y
-----------------------------------------------
x -1.0057
(0.5258)
Constant 10.6136**
(1.8813)
-----------------------------------------------
Observations 5
R2 0.5494
Adjusted R2 0.3992
Residual Std. Error 3.1198 (df = 3)
F Statistic 3.6578 (df = 1; 3)
===============================================
Note: *p<0.1; **p<0.05; ***p<0.01d. Practice
Now that you know how to use covariance, correlation coefficients, and linear regression to describe relationships in data, let’s apply what you’ve learned to the Covid19.csv dataset to compute the covariance, correlation coefficient, and regression line between dem16vs and mask_always.
- What is the covariance and correlation coefficient between
dem16vsandmask_always? Please compute them in a manual way.
## Covariance
## Correlation- What are the intercepts and slopes for the regression line between
dem16vsandmask_always? Note that in this practice, we are calculating the regression line based on the 500 randomly selected counties!
## Please find a and b! - Now you have the regression line, please add the line on the scatter plot using
geom_abline()!
## Add the regression line! Download .qmd for Discussion Section III
Discussion Section IV
Note: These materials were adapted from content prepared by Professors Yu-Shiuan (Lily) Huang (National Chengchi University) and Ryan Hubert (London School of Economics).
2. Learning from Data I
Note: Most of the materials for today’s discussion can be referred to STAT 414: Introduction to Probability Theory..
Data Generating Process (DGP)
The contemporary world is full of data and, especially, it is becoming easier and easier to transform it into data. Data analysis is about making good use of data for certain purposes, such as to predict or to make inferences about the world. For these tasks, a first step is to have a sound understanding on how the data was generated, i.e., which processes lead to its creation. The analytical strategy for data analysis will be highly dependent on which processes have generated the data under scrutiny.
Overall, in the following method sequence classes, we will only consider the type of probability based (stochastic) data generation processes. In the probability based DGP, data is generated by a probabilistic process, i.e., each element of the population is selected with a (known) probability to form part of the sample data. One possible way to achieve this is by using a random mechanism when generating the data.
Example:
Research Question: What percentage of U.S. adults support the death penalty?
Steps in Statistical Investigation:
Produce Data: Determine what to measure, then collect the data. The poll selected 1,082 U.S. adults at random. Each adult answered this question: “Do you favor or oppose the death penalty for a person convicted of murder?”
Explore the Data: Analyze and summarize the data.In the sample, 65% favored the death penalty.
Draw a Conclusion: Use the data, probability, and statistical inference to draw a conclusion about the population.
However, there is a cost that has to be taken into account: by drawing a sample we have not observed all of the individuals in the population. It is necessary, then, to account for the uncertainty implied in the sampling process that generated the data. In addition to the sampling uncertainty, there are other sources of uncertainty that would lead us to assume our data is generated by a stochastic DGP: theoretical and fundamental uncertatinty.
Let’s consider a scenario where we are not only interested in understanding the percentage of adults who support the death penalty in the U.S. but also whether there is a difference in support based on political affiliation, particularly Republicans. In this case, we have a theoretical framework that focuses on the relationship between a person’s political alignment and their stance on the death penalty. However, it’s possible that other factors, such as how individuals value a person’s life, could also influence their views on the death penalty. Unfortunately, we didn’t include these additional factors in our theory, which could potentially impact our data collection. Consequently, the results derived from our sample data might be subject to theoretical uncertainty. Additionally, there is the possibility of fundamental uncertainty within our sample data. This could arise when respondents pay minimal attention to the survey questions and opt for random responses regarding their opinions on the death penalty. Such factors are beyond the control of researchers and can introduce further uncertainty into our findings.
In summary, statistical inferences always involve an element of uncertainty. As a researcher, our aim is to account for this uncertainty by incorporating probability statements as an integral part of our conclusions in our findings.
Probability Distribution
How do we model a stochastic data generating process? Any time we want to answer a research question that involves using a sample to draw a conclusion about some larger population, we need to answer the question “how likely is it that we get a sample proportion of people supporting for the death penalty at 65%?” or “what is the probability of…?”. To answer such a question, we need to use probability to quantify the uncertainty of the outcomes of interest.
Just a reminder, when we employ a probability distribution to model a stochastic DGP, we are essentially playing God, and saying that the data for the variable of interest is generated by this probability distribution. However, in real-life, what we can directly observe is the empirical distribution (or frequency distribution) derived from our available sample data. We make an assumption that this empirical distribution from the sample data reflects the underlying theoretical probability distribution.
- Discrete Random Variables
What is a Random Variable \(X\)?
Given a random experiment with sample space \(S\), a random variable \(X\) is a set function that assigns one and only one real number to each element \(s\) that belongs in the sample space \(S\). The set of all possible values of the random variable \(X\), denoted \(x\), is called the support, or space, of \(X\). For example:
A rat is selected at random from a cage of male (\(M\)) and female rats (\(F\)). Once selected, the gender of the selected rat is noted. The sample space is thus: \[S = \left \{ M, F \right \}\]
Let’s further define the random variable \(X\) as follows:
- Let \(X = 0\) if the rat is male.
- Let \(X = 1\) if the rat is female.
Note that the random variable \(X\) assigns one and only one real number (0 and 1) to each element of the sample space (\(M\) and \(F\)). The support, or space, of \(X\) is \(\left \{ 0, 1 \right \}\).
Also note that we don’t necessarily need to use the number 0 and 1 as the support. For example, we could have alternatively (and perhaps arbitrarily) used the numbers 5 and 15, respectively. In that case, our random variable would be defined as \(X = 5\) of the rate is male, and \(X = 15\) of the rate is female.
What is the Probability Mass Function (PMF) of a discrete random variable?
The probability that a discrete random variable \(X\) takes on a particular value \(x\), that is, \(P(X = x)\), is frequently denoted \(f(x)\). The function \(f(x)\) is typically called the probability mass function (PMF).
The probability mass function, \(P(X = x) = f(x)\), of a discrete random variable \(X\) is a function that satisfies the following properties:
\(P(X = x) = f(x) > 0\) if \(x \in\) the support \(S\)
For every element in the support \(S\), all of the probabilities must be positive. Note that if \(x\) does not belong in the support \(S\), then \(f(x) = 0\).
\(\sum_{x \in S}^{}f(x)=1\)
If you add up the probabilities for all of the possible \(x\) values in the support \(S\), then the sum must equal 1.
\(P(X \in A) = \sum_{x \in A}^{}f(x)\)
To determine the probability associated with the event \(A\), you just sum up the probabilities of the \(x\) values in \(A\).
Exercise 1:
I toss a fair four-sided dice, and let \(X\) be defined as the outcome on the die I observe, \(\left \{ 1, 2, 3, 4 \right \}\). What is the sample space of \(X\), the possible outcomes of \(X\), as well as its probability mass function \(f(x)\)?
We can also plot PMF in R.
x <- 1:4 n <- length(x) fx <- rep(1/4, n)plot(x = x, y = fx, pch = 19, xaxt="n", ylim = c(0, 0.5), xlab = "x", ylab = "f(x)", main = "Probability Mass Function") axis(1, at = seq(min(x), max(x), by = 1))
What is the Cumulative Distribution Function (CDF) of a discrete random variable?
The cumulative distribution function (CDF) of the discrete random variable \(X\) has the following definition:
\[F(x) = P(X \leq x) = \sum_{a \leq x}^{}f(a)\]
in which \(F(x)\) is a non-decreasing step function.
We can also plot CDF in R. Let’s plot the CDF of the Exercise 1.
x <- 1:4 n <- length(x) fx <- rep(1/4, n) Fx <- cumsum(fx)## Let's make an empty plot plot(x = NA, y = NA, pch = NA, xaxt="n", xlim = c(min(x), max(x)), ylim = c(0, 1), xlab = "x", ylab = "F(x)", main = "Cumulative Distribution Function") axis(1, at = seq(min(x), max(x), by = 1)) ## Add closed circles, open circles, and finally the horizontal lines points(x = x, y = Fx, pch = 19) ## closed circles points(x = x[-1], y = Fx[-n], pch = 1) ## open circles for(i in 1:(n-1)) { points(x = x[i+0:1], y = Fx[c(i,i)], type = "l") }
Exercise 2:
Suppose \(X\) is a discrete random variable. Let the PMF of \(X\) be equal to:
\[f(x) = \frac{5-x}{10}, x = 1, 2, 3, 4. \]
The CDF of \(X\) is \(F(x) = P(X \leq x) = \sum_{a \leq x}^{}f(a)\).
2.1 What is the cumulative probability when \(X \leq 2\)?
2.2 Is \(P(X \leq 2)\) equal to \(P(X = 2)\)?
Continuous Random Variables
A continuous random variable takes on an uncountably infinite number of possible values. For a discrete random variable \(X\) that takes on a finite or countably infinite number of possible values, we determined \(P(X=x)\) for all of the possible values of \(X\), and called it the probability mass function (PMF).
For continuous random variables, the probability that \(X\) takes on any particular value \(x\) is 0. That is, finding \(P(X=x)\) for a continuous random variable \(X\) is not going to work. Instead, we’ll need to find the probability that \(X\) falls in some interval \((a, b)\), that is, we’ll need to find \(P(a<X<b)\). We’ll do that using a probability density function (PDF).
What is the Probability Density Function (PDF) of a continuous random variable?
The probability density function of a continous random variable \(X\) with support \(S\) is an integrable function \(f(x)\) satisfying the following:
\(f(x)\) is positive everywhere in the support \(S\), that is, \(f(x)>0\), for all \(x\) in \(S\).
The area under the curve f(x) in the support \(S\) is 1, that is:
\[\int_{S}^{}f(x)dx=1\]
If \(f(x)\) is the PDF of \(x\), then the probability that \(X\) belongs to \(A\), where \(A\) is some interval, is given by the integral of \(f(x)\) over that interval, that is:
\[P(X \in A) = \int_{A}^{}f(x)dx\]
As you can see, the definition for the PDF of a continuous random variable differs from the definition for the PMF of a discrete random variable by simply changing the summations that appeared in the discrete case to integrals in the continuous case.
Exercise 3:
Let \(X\) be a continuous random variable whose PDF is:
\(f(x) = 3x^2\), \(0<x<1\)
First, note again that \(f(x) \neq P(X=x)\). For example, \(f(0.9)=3(0.9)^{2}=2.43\), which is clearly not a probability! In the continuous case, \(f(x)\) is instead the height of the curve at \(X=x\), so that the total area under the curve is 1. In the continuous case, it is areas under the curves that define the probabilities.
Now let’s first start by verifying that \(f(x)\) is a valid probability density function.
\(f(x) > 0\) at any value of \(x\).
\(\int_{0}^{1}3x^2dx=x^3 |_{x=0}^{x=1}=1^3-0^3=1\)
3.1 What is the probability that \(X\) falls between \(\frac{1}{2}\) and 1? That is, what is \(P(\frac{1}{2}<X<1)\)?
3.2 Please use integration to show that \(P(X=\frac{1}{2})=0\).
Exercise 4:
Let \(X\) be a continuous random variable whose probability density function is:
\[f(x) = \frac{x^3}{4}\]
for any interval \(0<x<c\). What is the value of the constant \(c\) that makes \(f(x)\) a valid probability density function?
What is the Cumulative Distribution Function (CDF) of a continuous random variable?
You might recall that the cumulative distribution function is defined for discrete random variables as:
\[F(x) = P(X \leq x) = \sum_{a \leq x}^{}f(a)\]
Again, \(F(x)\) accumulates all of the probability less than or equal to \(x\).The cumulative distribution function for continuous random variables is just a straightforward extension of that of the discrete case. All we need to do is replace the summation with an integral.
The cumulative distribution function of a continuous random variable \(X\) is defined as:
\[F(x) = P(X \leq x) = \int_{a \leq x}^{}f(a)da\]
Recall that for discrete random variables, \(F(x)\) is in general a non-decreasing step function. For continuous random variables, \(F(x)\) is a non-decreasing continuous function.
Exercise 5:
Let’s return to the example in which \(X\) has the following probability density function:
\(f(x) = 3x^2\), \(0<x<1\)
What is the cumulative distribution function \(F(x)\)?
Multivariate Probability Distributions
Now we know the probability distribution of both discrete and continuous random variables, respectively. Let’s extend the concept of a probability distribution of one random variable \(X\) to a joint probability distribution of two discrete random variables \(X\) and \(Y\). For example, suppose \(X\) denotes the number of significant others a randomly selected person has, and \(Y\) denotes the number of arguments the person has each week. We might want to know if there is a relationship between \(X\) and \(Y\). Or, we might want to know the probability that \(X\) takes on a particular value \(x\) and \(Y\) takes on a particular value \(y\). That is, we might want to know \(P(X=x, Y=y)\).
Suppose we toss a pair of fair, four-sided dice, in which one the dice is red and the other is black. We’ll let:
- \(X\) = the outcome on the red die = \(\left \{ 1, 2, 3, 4 \right \}\)
- \(Y\) = the outcome on the black die = \(\left \{ 1, 2, 3, 4 \right \}\)
What is the support of \(f(x, y)\)? What is the probability that \(X\) takes on a particular value \(x\), and \(Y\) takes on a particular value \(y\)? That is, what is \(P(X=x, Y=y)\)? Please calculate the probability for each cell in the table below. Note that columns represent the outcomes of the black die, whereas rows represent the outcomes of the red die.
\(f(x,y)\) \(Y=1\) \(Y=2\) \(Y=3\) \(Y=4\) \(f_X(x)\) \(X=1\) \(X=2\) \(X=3\) \(X=4\) \(f_Y(y)\) Now that we’ve found our first joint probability mass function, let’s formally define it now.
Let \(X\) and \(Y\) be two discrete random variables, and let \(S\) denote the two-dimensional support of \(X\) and \(Y\). Then, the function \(f(x, y) = P(X=x, Y=y)\) is a joint probability mass function if it satisfies the following three conditions:
\(0 \leq f(x, y) \leq 1\)
Each probability must be a valid probability number between 0 and 1 (inclusive).
\(\sum_{}^{}\sum_{(x, y) \in S}^{}f(x, y)=1\)
The sum of the probabilities over the entire support \(S\) must equal 1.
\(P[(X, Y)\in A]=\sum_{}^{}\sum_{(x, y)\in A}^{}f(x, y)\) where \(A\) is a subset of \(S\)
In order to determine the probability of an event \(A\), you simply sum up the probabilities of the \((x, y)\) values in \(A\).
Marginal Distribution
Now, if you take a look back at the representation of our joint p.m.f. in tabular form, you can see that the last column contains the probability mass function of \(X\) alone, and the last row contains the probability mass function of \(Y\) alone. Those two functions, \(f(x)\) and \(f(y)\), which in this setting are typically referred to as marginal probability mass functions, are obtained by simply summing the probabilities over the support of the other variable. That is, to find the probability mass function of \(X\), we sum, for each \(x\), the probabilities when $y=1, 2, 3, $ and \(4\). That is, for each \(x\), we sum \(f(x, 1)\), \(f(x, 2)\), \(f(x, 3)\), and \(f(x, 4)\). Now that we’ve seen the two marginal probability mass functions in our example, let’s give a formal definition of a marginal probability mass function.
Let \(X\) be a discrete random variable with support \(S_1\), and let \(Y\) be a discrete random variable with support \(S_2\), Let \(X\) and \(Y\) have the joint probability mass function \(f(x, y)\) with support \(S\). Then, the probability mass function of \(X\) alone, which is called the marginal probability mass function of \(X\), is defined by:
\[f_X(x)=\sum{y}{}f(x, y)=P(X=x)\] \[x \in S_1\]
where, for each \(x\) in the support \(S_1\), the summation is taken over all possible values of \(y\). Similarly, the probability mass function of \(Y\) alone, which is called the marginal probability mass function of \(Y\), is defined by:
\[f_Y(y)=\sum{x}{}f(x, y)=P(Y=y)\] \[x \in S_2\]
where, for each \(y\) in the support \(S_2\), the summation is taken over all possible values of \(x\).
If you again take a look back at the representation of our joint p.m.f. in tabular form, you might notice that the following holds true:
\[P(X=x, Y=y) = \frac{1}{16} = P(X=x) \cdot P(Y=y) = \frac{1}{4} \cdot \frac{1}{4} = \frac{1}{16}\]
for all \(x \in S_1\), \(y \in S_2\). When this happens, we say that \(X\) and \(Y\) are independent. A formal definition of the independence of two random variables \(X\) and \(Y\) follows.
The random variables \(X\) and \(Y\) are independent if and only if:
\[P(X=x, Y=y) = P(X=x) \times P(Y=y)\]
for all \(x \in S_1\), \(y \in S_2\). Otherwise, \(X\) and \(Y\) are said to be dependent.
Exercise 6:
Please find the marginal distributions of \(X\) and \(Y\), respectively.
\(f(x, y)\) \(X=0\) \(X=1\) \(X=2\) \(f_Y(y)\) \(Y=1\) \(0.1\) \(0.2\) \(0.3\) \(Y=2\) \(0.05\) \(0.15\) \(0.2\) \(f_X(x)\) Is \(X\) and \(Y\) independent from each other or not?
Conditional Distribution
Marginal distributions tell us the distribution of a particular variable ignoring the other variable(s). In contrast, the conditional distribution of \(X\) given \(Y\) gives you a distribution over \(X\) when holding \(Y\) fixed at specific values. Formally:
\[f_{X|Y}(x|y)=\frac{f(x, y)}{f_Y(y)}\]
The conditional distribution of \(Y\) given \(X\) is defined by:
\[f_{Y|X}(y|x)=\frac{f(x, y)}{f_X(x)}\]
Recall the example we had for joint distribution.
\(f(x,y)\) \(Y=1\) \(Y=2\) \(Y=3\) \(Y=4\) \(f_X(x)\) \(X=1\) \(\frac{1}{16}\) \(\frac{1}{16}\) \(\frac{1}{16}\) \(\frac{4}{16}\) \(\frac{4}{16}\) \(X=2\) \(\frac{1}{16}\) \(\frac{1}{16}\) \(\frac{1}{16}\) \(\frac{1}{16}\) \(\frac{4}{16}\) \(X=3\) \(\frac{1}{16}\) \(\frac{1}{16}\) \(\frac{1}{16}\) \(\frac{1}{16}\) \(\frac{4}{16}\) \(X=4\) \(\frac{1}{16}\) \(\frac{1}{16}\) \(\frac{1}{16}\) \(\frac{1}{16}\) \(\frac{4}{16}\) \(f_Y(y)\) \(\frac{4}{16}\) \(\frac{4}{16}\) \(\frac{4}{16}\) \(\frac{4}{16}\) \(1\) What is the conditional distribution of \(Y\) given \(X=2\)?
When \(Y=1\): \(f_{Y|X}(y|x=2)=\frac{f(x, y)}{f_X(x=2)}=\frac{\frac{1}{16}}{\frac{4}{16}}=\frac{1}{4}\)
When \(Y=2\): \(f_{Y|X}(y|x=2)=\frac{f(x, y)}{f_X(x=2)}=\frac{\frac{1}{16}}{\frac{4}{16}}=\frac{1}{4}\)
When \(Y=3\): \(f_{Y|X}(y|x=2)=\frac{f(x, y)}{f_X(x=2)}=\frac{\frac{1}{16}}{\frac{4}{16}}=\frac{1}{4}\)
When \(Y=4\): \(f_{Y|X}(y|x=2)=\frac{f(x, y)}{f_X(x=2)}=\frac{\frac{1}{16}}{\frac{4}{16}}=\frac{1}{4}\)
So,
\(f_{Y|X}(y|x=2)= \frac{1}{4} if y = 1, 2, 3, 4\)
We will get the same answers for the conditional probability of \(Y\) given \(X=1\), \(X=3\), or \(X=4\).
From this example of finding the conditional probability of \(Y\) given \(X=2\), we again find that \(X\) and \(Y\) are independent from each other, as the distribution over \(Y\) does not change as \(Y\) changes. That is,
\[f(x, y) = f_X(x)\cdot f_Y(y) \Rightarrow f_{X|Y}{x|y} = f_X(x) and f_{Y|X}{y|x} = f_Y(y)\]
Exercise 7:
Find the conditional distribution of \(X\) given \(Y=2\).
\(f(x, y)\) \(X=0\) \(X=1\) \(X=2\) \(f_Y(y)\) \(Y=1\) \(0.1\) \(0.2\) \(0.3\) \(0.6\) \(Y=2\) \(0.05\) \(0.15\) \(0.2\) \(0.4\) \(f_X(x)\) \(0.15\) \(0.35\) \(0.5\) \(1\) Central Tendency of Probability Distributions
The expected value, or mathematical expectation, of a discrete random variable \(X\) is:
\[E(X)=\sum_{x\in X}{}xf(x)\]
where \(x\) is the possible outcomes of the discrete random variable \(X\), and \(f(x)\) is the probability mass function of \(X\).
For example, what is the average toss of a fair six-sided die?
If the random variable \(X\) is the top face of a tossed, fair, six-sided die, then the PMF of \(X\) is:
\[f(x) = \frac{1}{6}\]
for \(x = 1, 2, 3, 4, 5,\) and \(6\). Therefore, the average toss, that is the expected value of \(X\), is:
\[E(X) = 1(\frac{1}{6}) + 2(\frac{1}{6}) + 3(\frac{1}{6}) + 4(\frac{1}{6}) + 5(\frac{1}{6}) + 6(\frac{1}{6}) = 3.5\]
Hmm… if we toss a fair, six-sided die once, should we expect the toss to be 3.5? No, of course not! All the expected value tells us is what we would expect the average of a large number of tosses to be in the long run. If we toss a fair, six-sided die a thousand times, say, and calculate the average of the tosses, will the average of the 1000 tosses be exactly 3.5? No, probably not! But, we can certainly expect it to be close to 3.5. It is important to keep in mind that the expected value of is a theoretical average, not an actual, realized one! Let’s run a simulation to observe this more directly.
x <- 1:6 s1 <- sample(x, replace = TRUE, size = 100) mean(s1)[1] 3.25s2 <- sample(x, replace = TRUE, size = 1000) mean(s2)[1] 3.45s3 <- sample(x, replace = TRUE, size = 10000) mean(s3)[1] 3.508Expectations have some nice properties:
The expected value of a constant \(a\) is that constant: \(E(a) = a\).
The expected value of a constant \(b\) times a random variable \(X\) is that constant times the expected value of \(X\): \(E(bX) = bE(X)\).
Given two random variables, \(X\) and \(Y\), \(E(X+Y) = E(X) + E(Y)\).
\(E(X)\) always falls between the minimum and maximum values that \(X\) can take: \(min X \leq E(X) \leq max X\).
Exercise 8:
Suppose the PMF of the discrete random variable \(X\) is:
\(x\) \(0\) \(1\) \(2\) \(3\) \(f(x)\) \(0.2\) \(0.1\) \(0.4\) \(0.3\) 8.1 What is the expected value of \(X\), that is, \(E(X)\)?
8.2 Suppose that there is another discrete random variable \(Y\), and the expected value of \(Y\) is \(3.2\), that is, \(E(Y) = 3.2\). What is the expected value of \(2X+3Y\), that is, \(E(2X+3Y)\)?
Exercise 9:
9.1 Find the expected values of the random variables from their marginal distributions.
9.2 Find the expected value of \(X\) given \(Y=2\).
\(f(x, y)\) \(X=0\) \(X=1\) \(X=2\) \(f_Y(y)\) \(Y=1\) \(0.1\) \(0.2\) \(0.3\) \(0.6\) \(Y=2\) \(0.05\) \(0.15\) \(0.2\) \(0.4\) \(f_X(x)\) \(0.15\) \(0.35\) \(0.5\) \(1\) All these nice properties can also be applied to a continuous random variable. The expected value of a continuous random variable is:
\[\int_{x\in X}{}xf(x)dx\]
Let’s use an example to practice. Let the random variable \(X\) denote the time a person waits for an elevator to arrive. Suppose the longest one would need to wait for the elevator is 2 minutes, so that the possible values of \(X\) (in minutes) are given by the interval \([0,2]\). A possible PDF for \(X\) is given by:
\[f(x) = \begin{cases} x, & \text{for } 0 < x \leq 1\\ 2-x, & \text{for } 1 < x \leq 2 \\ 0, & \text{otherwise } \end{cases} \] What is the expected value of \(X\)?
\[E(X) = \int_{0}^{1}x\cdot xdx + \int_{1}^{2}x\cdot(2-x)dx = \int_{0}^{1}x^2dx + \int_{1}^{2}(2x-x^2)dx = \frac{1}{3} + \frac{2}{3} = 1\]
Thus, we expect a person will wait 1 minute for the elevator on average.
Dispersion of Probability Distributions
The variance of a random variable \(X\) is:
\[ Var(X) = E((X-E(X))^2) = E(X^2) - E(X)^2\]
The formulat can be applied to both discrete and continuous random variables.
The variance of a random variable also has nice properties:
The variance of a constant \(a\) is \(0\): \(Var(a) = 0\).
\(Var(X)\) is always weakly positive: \(Var(X) \geq 0\).
The variance of a constant \(b\) times a random variable \(X\) is that constant squared times the variance of \(X\): \(Var(bX) = b^2 Var(X)\).
If \(X\) and \(Y\) are stochastically independent, then \(Var(X+Y) = Var(X) + Var(Y)\).
Exercise 10:
10.1 What is the variance of \(X\)? 10.2 What is Var(2X)?
\(x\) \(0\) \(1\) \(2\) \(3\) \(f(x)\) \(0.2\) \(0.1\) \(0.4\) \(0.3\) Exercise 11:
Recall the elevator question, please compute the variance of \(X\).
\[f(x) = \begin{cases} x, & \text{for } 0 < x \leq 1\\ 2-x, & \text{for } 1 < x \leq 2 \\ 0, & \text{otherwise } \end{cases} \]