STA4234: Regression Analysis

Course Description

STA4234 Regression Analysis is the workhorse course of applied statistics — the one that turns "these two things seem related" into a model with estimated coefficients, stated assumptions, and an honest account of what it can and cannot support.

The statewide inventory records the course at Florida Atlantic University, Florida Gulf Coast University, Florida International University and the University of West Florida. ⚠ Three full descriptions were retrievable and they agree on the subject.

Institution	Title	Description and prerequisites
UWF	Regression Analysis (matches the statewide title)	"Simple Linear Regression, Multiple Linear Regression, Model Adequacy Checking, Transformations and Weighting to Correct Model Inadequacies, Diagnostics for Leverage and Influence, Polynomial Regression Models, Indicator Variables, Multicollinearity, Variable Selection and Model Building, Validation of Regression Models, Introduction to Logistic Regression." Department of Mathematics and Statistics, 3 sh. Prerequisite: STA 2023 OR STA 3162C.
FGCU	Intro. to Regression Analysis	"Study of theory and applications of regression analysis. Topics include: general linear model, parameter estimation, residual analysis, polynomial and logarithmic regression, model identification, applications to biological and social sciences." 3 credits. ⚠ Prerequisites: (STA 2023 or STA 2037) AND (MAC 2311 or MAC 2233) — a calculus requirement UWF does not impose.
FIU	Introduction to Regression	"Multiple and polynomial regression, residual analysis, model identification and other related topics." ⚠ "Credit will not be given for both STA 4234 and STA 5236."

What the course actually does. Regression fits a model of the form "this outcome, as a function of these predictors, plus error" — and ⚠ almost everything in the course is about the last term. Estimating the coefficients is arithmetic a computer does instantly. Knowing whether the estimates mean anything is the discipline.

The assumptions, which are the course's spine. Linearity in the parameters; independence of errors; constant error variance (homoscedasticity); and approximate normality of errors for inference. ⚠⚠ Each has a diagnostic, each has a consequence when violated, and each has a remedy — which is why UWF's topic list runs model adequacy checking and transformations and weighting to correct model inadequacies back to back. The residual plot is the single most informative object in applied statistics and students under-use it badly.

The four problems that recur in every real dataset.

• Outliers, leverage and influence — ⚠ and these are three different things. An outlier has a large residual; a high-leverage point has an unusual predictor value; an influential point is one whose removal changes the fitted model. A point can be any one without being the others, and Cook's distance exists to identify the third.
• Multicollinearity — predictors correlated with each other. ⚠ It does not bias the predictions; it inflates the standard errors, which makes individual coefficients unstable and their interpretation unreliable. A model can predict well and tell you nothing about which predictor matters.
• Model selection — ⚠⚠ where the course's deepest lesson lives. Stepwise procedures are taught, and their problems are taught with them: p-values computed after a search over models are not valid p-values, and the resulting model is fitted partly to noise. This is the same problem as overfitting in machine learning, arrived at from the statistical side.
• Validation — UWF names it explicitly. Does the model hold up on data it was not fitted to?

⚠⚠ And the sentence the whole course exists to earn the right to say: regression estimates association, and association is not causation. A coefficient is the expected difference in the outcome per unit difference in that predictor, holding the others in the model fixed — which is a statement about the model, not about what would happen if you intervened. Omitted variables, reverse causation and selection all produce real coefficients that mean nothing causal.

Learning Outcomes

Required Outcomes

• Formulate a simple linear regression model and estimate its parameters by least squares.
• Interpret slope and intercept correctly in the context of the data.
• Conduct inference on regression parameters — hypothesis tests and confidence intervals.
• Construct and distinguish confidence intervals for the mean response and prediction intervals for a new observation.
• Interpret R², adjusted R² and the standard error of the estimate, and explain the limits of each.
• Formulate and fit a multiple linear regression model.
• Interpret partial regression coefficients and explain what "holding the others constant" means and does not mean.
• Conduct the overall F test and partial F tests for subsets of predictors.
• State the assumptions of the linear regression model and their role in estimation and inference.
• ⚠ Perform residual analysis and diagnose violations of assumptions from plots.
• Apply transformations and weighted least squares to correct model inadequacies.
• Identify outliers, high-leverage points and influential observations, and distinguish them.
• Compute and interpret influence diagnostics — hat values, studentised residuals, Cook's distance, DFFITS.
• Detect multicollinearity using variance inflation factors and explain its consequences.
• Fit and interpret polynomial regression models, and explain the risks of high-order terms.
• Use indicator (dummy) variables to include categorical predictors, and interpret their coefficients.
• Fit and interpret interaction terms.
• Apply variable selection procedures and information criteria, and explain their limitations.
• ⚠ Explain why inference after model selection is not valid in the usual way.
• Validate a regression model on held-out or new data.
• Fit and interpret a logistic regression model for a binary outcome, including odds ratios.
• Use statistical software to fit models and read its output correctly.
• ⚠ Explain the distinction between association and causation in a regression context.
• Communicate a regression analysis in writing to a non-statistical audience.

Optional Outcomes

• Derive least squares estimators using the matrix formulation of the general linear model.
• Prove properties of the estimators, including the Gauss–Markov theorem.
• Apply generalised linear models beyond logistic regression.
• Apply ridge, lasso and other penalised regression methods.
• Apply nonlinear regression.
• Apply mixed or hierarchical models.
• Explain time series regression and autocorrelated errors.
• Apply bootstrap and resampling methods.
• Explain the connection between regression and machine learning methods.

Major Topics

Required Topics

• Simple linear regression — estimation and inference.
• Confidence and prediction intervals.
• Multiple linear regression.
• The general linear model and parameter estimation.
• Model assumptions.
• Residual analysis and model adequacy checking.
• Transformations and weighted least squares.
• Leverage and influence diagnostics.
• Multicollinearity.
• Polynomial regression.
• Indicator variables and interactions.
• Variable selection and model building.
• Model validation.
• Logistic regression.
• Statistical software and output interpretation.
• Communicating results.

Optional Topics

• Matrix algebra of regression; Gauss–Markov.
• Generalised linear models.
• Penalised regression — ridge, lasso, elastic net.
• Nonlinear regression.
• Mixed and hierarchical models.
• Time series and autocorrelation.
• Bootstrap methods.
• Causal inference in outline.
• Regression and machine learning.

Resources & Tools

• Montgomery, Peck and Vining, Introduction to Linear Regression Analysis — ⚠⚠ the standard text for this course, and UWF's topic list follows its chapter order almost exactly. If you can identify one likely assigned book, it is this.
• Kutner, Nachtsheim, Neter and Li, Applied Linear Statistical Models — comprehensive and heavy; Weisberg, Applied Linear Regression; Fox, Applied Regression Analysis and Generalized Linear Models for the social science orientation.
• ⚠⚠ Free and outstanding: James, Witten, Hastie and Tibshirani, An Introduction to Statistical Learning — free as a PDF from the authors, with both R and Python editions, and its regression and model-selection chapters are the clearest available treatment of the material this course covers, written from the prediction side. Read chapters 3 and 6 alongside whatever is assigned. The same group's Elements of Statistical Learning is the advanced version, also free.
• Also free: Faraway, Practical Regression and Anova using R; OpenIntro Statistics for revision of the introductory material; and StatQuest's videos, which are unusually good on residuals, R² and logistic regression.
• Software: ⚠ R — free, and the language regression is taught in most often; lm(), glm(), car and broom are the working tools; Python with statsmodels (⚠ use statsmodels rather than scikit-learn for this course — it produces the inferential output the course is about, which scikit-learn deliberately omits); SAS, still standard in pharmaceutical and government work; SPSS, Minitab and JMP where the department licenses them; JASP is free and friendly.
• ⚠ Learn to read software output rather than to memorise it. Every package prints the same quantities in a different arrangement — coefficient, standard error, t statistic, p-value, and the model-level F and R². Recognising them anywhere is the transferable skill.
• Data for a real project, free: the UCI Machine Learning Repository; Kaggle; the US Census and American Community Survey; data.gov; and ⚠ Florida-specific sources — Florida Health CHARTS, the Florida Department of Education, FDOT traffic data and the water management districts — which make a locally relevant project entirely feasible.
• Professional: the American Statistical Association and its student chapters; ⚠ the ASA's Ethical Guidelines for Statistical Practice are free, short and directly relevant to this course's honesty problems.

Career Pathways

⚠⚠ This is among the most directly employable single courses in a mathematics or statistics degree, because regression is the shared language of quantitative work across fields.

• Statisticians (SOC 15-2041) — ⚠ a master's is normal for the title; this course is core to it.
• Data scientists (SOC 15-2051) — ⚠⚠ and the connection is closer than students expect: linear and logistic regression are the baseline models in machine learning, and a great deal of production modelling is regression done carefully.
• Data analysts and business intelligence analysts (SOC 15-2051, 13-1111).
• Biostatisticians and clinical trial analysts (SOC 15-2041) — ⚠ Florida's academic medical centres, Moffitt Cancer Center and the pharmaceutical and CRO sector.
• Actuaries (SOC 15-2011) — ⚠ regression is examined in the Society of Actuaries' Statistics for Risk Modeling exam; Florida's insurance industry is large and concentrated.
• Economists and econometricians (SOC 19-3011) — ⚠ econometrics is regression with the causal problems taken seriously.
• Market research analysts (SOC 13-1161).
• Operations research analysts (SOC 15-2031).
• Epidemiologists and public health analysts (SOC 19-1041) — logistic regression is the field's standard tool.
• Quality and reliability engineers (SOC 17-2112) — ⚠ Montgomery's textbook comes from this tradition.
• Sports analytics (SOC 15-2051) — ⚠ a real Florida sector given the professional teams and college athletics.
• Credit risk and quantitative finance (SOC 13-2054, 13-2099) — ⚠ logistic regression remains standard in credit scoring precisely because it is explainable, which regulators require.

Special Information

⚠⚠ Check the prerequisite — it tells you the course

Institution	Prerequisite	What it implies
UWF	STA 2023 OR STA 3162C	introductory statistics only — an applied, procedure-oriented course
FGCU	(STA 2023 or STA 2037) AND (MAC 2311 or MAC 2233)	⚠ plus calculus — theory as well as applications, and likely the matrix formulation

• ⚠ Unlisted but genuinely valuable either way: linear algebra. The general linear model is matrix algebra, and a student who has met matrices, rank and projections finds the theoretical half far easier. It is rarely required and it always helps.
• Also assumed: fluency with the introductory course's inference — sampling distributions, confidence intervals, hypothesis tests and p-values. ⚠ If those are hazy, revise them before the term rather than during it, because regression inference is built directly on them.
• Prior software experience is not required and is a genuine advantage. ⚠ If your course uses R and you have never used it, work through a short introduction before week one — students who are simultaneously learning R and learning regression have a harder term than they need to.

⚠ FIU's exclusion

Credit will not be given for both STA 4234 and STA 5236, its graduate counterpart. ⚠ Relevant if you are considering taking the graduate version for a heavier treatment, or arriving with one already completed. Check before registering.

Course format and workload

3 credits, 45 contact hours — lecture, three hours per week, ⚠ normally with substantial computing outside class even where no laboratory is scheduled. UWF notes it may not be repeated for credit.

Expect 7–10 hours per week outside class. Assessment typically includes examinations, computing assignments producing fitted models and written interpretation, and frequently a data analysis project on a dataset of your choosing.

⚠⚠ The project is the course's most valuable artefact. Choose a real dataset and a real question, and keep the write-up — a clean, honest regression analysis with diagnostics and stated limitations is exactly what an employer wants to see, and it is rarer among applicants than it should be.

⚠ Where students struggle

• ⚠⚠ Interpreting coefficients. The characteristic error. "Holding the other variables in the model constant" is a statement about the model's arithmetic, not about an experiment you could run — and where predictors are correlated, holding one constant while another varies may describe no real situation at all.
• ⚠⚠ Treating R² as a grade. A high R² does not mean the model is correct, useful, or causally meaningful; a low one does not mean the analysis failed. R² measures variance explained in this sample, it never decreases when predictors are added, and that is why adjusted R² exists.
• Skipping diagnostics. ⚠ The software will fit a model to anything, including data that violates every assumption. It prints no warning. Plot the residuals, every time.
• Confusing outliers, leverage and influence. Three different diagnostics for three different problems.
• Deleting inconvenient points. ⚠⚠ An influential point is information, not an obstacle. Removing observations to improve a fit, without a substantive reason and without saying so, is a form of dishonesty — and it is easy to do without noticing you have done it.
• Stepwise regression treated as automatic model building. ⚠ The procedure is taught and its problems are taught with it: it capitalises on chance, the reported p-values are not valid, and it substitutes an algorithm for subject-matter knowledge.
• Extrapolation. Predicting outside the range of the observed predictors, where the model has no evidence at all.
• Causal language creeping into the write-up. ⚠ "Increasing X by one unit increases Y by β" is a causal claim, and the analysis usually does not support it.

⚠ Statistical ethics, which this course is the right place for

Regression is where most quantitative dishonesty happens, usually without intent.

• ⚠⚠ "p-hacking" and the garden of forking paths. Fitting many models and reporting the one that reached significance produces a p-value that means nothing. This is a documented cause of the replication problems across several research fields, and it is done far more often by accident than by design.
• Pre-specification and transparency. ⚠ Reporting what you did, including what you tried and discarded, is the correction — and it is what the ASA's ethical guidelines require.
• Consequences. Regression models decide credit, insurance pricing, hiring screens and resource allocation. ⚠ A model fitted on historical outcomes reproduces historical patterns, including ones that should not be reproduced — and excluding a protected characteristic does not remove it if correlated predictors remain.

Articulation and transfer

The number STA4234 is used at all four institutions and SCNS articulation is clean; only the titles drift.

⚠ The practical risk is the prerequisite and depth gap, plus FIU's exclusion. Keep the syllabus.

This is a 4000-level upper-division course. ⚠ Florida College System institutions do not offer it, but STA2023 — the lower-division prerequisite — is taught everywhere and transfers cleanly. Take it before transferring.

Prefix note. STA is statistics; MAC, MAP and MAS the mathematics prefixes; QMB quantitative methods in business; ECO economics; PSY and SYA for the psychology and sociology statistics sequences. ⚠⚠ Regression is taught under STA, QMB (as business statistics or forecasting) and ECO (as econometrics), and inside psychology and sociology methods sequences. They cover overlapping material for different audiences with different notation, and a receiving programme frequently wants its own version. Search by subject rather than prefix and confirm with the department.

AI Integration

⚠⚠ This course sits at the historical root of machine learning, and saying so makes the material land better. Linear regression is the simplest supervised learning model; logistic regression is the simplest classifier; and the bias–variance trade-off, overfitting, regularisation and cross-validation all appear here in statistical dress. A student who understands regression properly has the conceptual foundation for machine learning and knows something most practitioners of it skipped.

Where AI assistance genuinely helps:

• Writing and debugging R or Python code — ⚠ genuinely useful, and not what the course is assessing.
• Explaining a concept a second way — multicollinearity, the meaning of leverage, why adjusted R² exists.
• Explaining software output field by field.
• Suggesting diagnostics to run on a fitted model.
• Improving the writing of a results section.

⚠⚠ Where it fails, and the failures are the course's own target errors:

• ⚠⚠ Interpretation, stated confidently and wrongly. Models routinely produce causal language for regression coefficients and describe p-values incorrectly — as the probability the null hypothesis is true, which it is not. These are exactly the misstatements the course exists to correct, and they are abundant in the text the model learned from.
• Skipping assumptions. ⚠ Asked to analyse data, a model will fit and report without checking anything — which is the behaviour the course is training out of you.
• Fabricated statistics and citations in a write-up.
• Plausible arithmetic that is wrong. Do the computation in software, which is definitive and free.
• Model selection without judgement. ⚠ A model asked for "the best model" will run a selection procedure and report the winner, with all the problems the course names. Subject-matter knowledge about which predictors belong is not available to it.

⚠ The professional point. Regression models are used in regulated settings — credit, insurance, employment, clinical research — where the analyst must be able to explain the model, defend its assumptions and state its limitations. "The software fitted it" is not an answer to a regulator, a reviewer or a court. ⚠⚠ Explainability is a large part of why regression remains in use where more flexible models predict better: a logistic regression coefficient can be explained to a customer denied credit, and a gradient-boosted ensemble usually cannot. That is a commercial and legal fact, not a statistical preference.

Academic integrity. Follow the course policy, which for computing assignments is normally specific. Submitting generated analysis as your own violates every Florida institution's policy — and in this course the deliverable employers want is a write-up you can defend line by line.