Hi everyone,

I have a treatment (i.e. a treatment dose), say X, that can take on values from 0–5 and is continuous in between, i.e. bounded continuous. A value of 0 will also be relatively common.

I’m specifying a treatment model for the conditional density given confounders L, i.e f(X|L) and an outcome model for Y~X to estimate the marginal E(Y^x).

My question is how to best treat the treatment variable 1) when it’s the outcome/dependent variable and 2) when it’s the exposure/independent variable in such scenario.

I don’t think I should be using model fit to pick a conditional distribution, and for interpretation sake I think assuming X has either a linear or quadratic relationship with Y is simplest. For the treatment model specifying a normal conditional density as long as the covariate balance is reasonable would be easiest I guess too.

Maybe someone has a better idea that doesn’t make interpretation too difficult, since it’s for a medical paper that people without strong statistical backgrounds will also read.

For a game, I'm simulating opening packs of cards, and my simulation results aren't quite holding up to the math that I've done.

The concept of the game is that you're opening packs of cards until you have a complete set, then you sell the set and can use the money to open more exclusive (and expensive) packs.

In each pack, there are 6 cards from a 'common' list, 3 cards from an 'uncommon' list, and a 50/50 shot at a 'rare' card. If the final card is not rare, it's drawn from the uncommon list.

There are 25 total commons, 10 uncommons, and 5 rares.

My understanding is that based on the stamp collector's problem, the expected number of trials to collect all cards from a set is equal to n Hn, where Hn is the nth harmonic number.

I've chosen to model the expected number of packs to open as MAX (E(T) common / 6, E(T) uncommon / 3.5, E(T) rare * 2).

The rationale is that there are six commons, so you'd naturally divide by E(T) common by 6 because you actually have 6 attempts in each card pack.

The reason I have chosen to take the maximum value of these three is that because you're drawing from all three lists at the same time, the expected number of packs should be equal to the hardest one of the three to complete.

The issue I have is that after running experiments (1M simulated packs), the average number of packs to get a full set is about 10% higher than the E(T) I've calculated.

Would somebody mind helping me learn where I've gone wrong?

Calculations:

E(T) common = 25/6 * H25 = 15.90

E(T) uncommon = 10/3.5 * H10 = 8.80

E(T) rare = 5*2 * H5 = 22.83

Overall E(T) should be 22.83

Experimental results:

avg: 24.702346837944663, max: 119, min: 8, med: 22, mode: 18 (2286 occurrences)

I'm a data scientist at a credit union. I have some DS experience elsewhere, but I'm still wet behind the ears, so to speak. Our data department is small and I'm the only one with DS knowledge. I know explainability to important when working in financial institutions because we get audited. Therefore, I'll be building a lot of logistic regression and decision trees in my future. That said, we are interested in understanding the potential impact of rate changes (fed and our own) on deposits, loan growth, etc as well as understanding our loan portfolio risk. My thought was to use bayesian modeling so we can understand the uncertainty. That said, I wasn't sure if this could cause issues when we're auditing, even though we wouldn't be using the model to impact customer decisions (approve loans, etc.). Does anyone have any advice on this? Thank you!

Hello,

I come from a background of applied math and recently I tried to dive deeper into stats. The frequentist view looked reasonable albeit a bit restrictive. And then there is the bayesian perspective, intuitively it "feels" reasonable as well since it's kind of intertwined with common sense, but I just can't wrap my head around it.

For example, I have no idea what the interpretation of probability is under this view. I mean what do I mean that an event is 60% vs 70% likely?

Also, I have a problem with the subjectivity of the view. There is this argument that if two observers have different beliefs about the same event, then at least one of them is wrong. How is this taken care of or circumvented perhaps?

Just had my first proper Stats class at college (I’m somewhere between a sophomore and a junior in terms of how much school I have left) and it kinda freaked me out. About 5-10 minutes into the professor introducing himself, he said, verbatim: “I like AI”. He went on to say he won’t “require” that we use AI (or maybe more accurately LLMs) but he more or less encouraged/endorsed it, and there was a quote in the syllabus from a bluesky poster that said “I expect you to write good code because gravity is lower now”, which to me reads “you have access to ChatGPT now so I’m gonna make things super hard to where you feel you have to use it”.

I’ve never gone out of my way to use any sort of generative AI/LLM (we had an LLM answer checker thing in my 200 level python class that we had to use in order to move on to the next question in our assignments, idk how much that counts. We weren’t allowed to use generative AI to answer questions for us in that same class) and I don’t have basically any interest in it, largely because of its environmental and human impact. I’ll be the first to admit I am not the most educated when it comes to the extent of said impact, but as it stands, I’d really rather not use it.

Am I overreacting? Like I said this was my first proper in-person stats class so it could’ve just been a bad first experience, but I also know that AI is becoming more and more prevalent. Thank you for any feedback or advice!

(Also sorry if this is not the right place for a question like this, please point me in the right direction if that’s the case)

Hi all, I've been working on a little personal project the last couple of days. I am a DnD player/DM, and I have 7 d20s (20 sided dice). I've been repeatedly rolling them and noting the results to see if any of them appear to be weighted in any one direction.

I ran a Chi-Square test for each of the dice, with some varying results, one die in particular stood out with a very low P value (0.0007), another was 0.3, another a tick above 0.5, the rest were solidly not significant.

But then I got thinking - due to the geometry of a 20 sided die, each face is not independent from the other - some faces are closer than others, so if, for example, a standard d20 was weighted towards 20, you would also expect to see higher rates of 2, 8 and 14, as they are right next to the 20 face and should also benefit (albeit to a lesser extent) from the weighting towards the 20. I'm not an expert, it's been about 15 years since I last studied any sort of statistics, but I believe this means a Chi-Squared test isn't actually an appropriate test.

To try to fix this I built a table recording the distance (in terms of number of faces) of each value from every other value, and based on that distance assigned a weighting of 1 (same value, distance 0) to 0 (opposite side of the die, distance of 5). That generates for any given die value an array of weightings that can be applied across all rolls (not just the rolls of that specific value) to produce an adjusted distribution that takes into account not just how many times that particular value was rolled, but also geometrically proximal (and distal) values, to try to identify any particular direction of weighting (basically using a SUMPRODUCT function to multiply the actual distribution of dice rolls against the weight values for the particular die value you are looking at, and repeating for each possible value across each die).

The effect of this, however, is that it has dramatically reduced the variance against the expected distribution, and my P values are now all 1. I find it difficult to believe that all 7 of my dice, after 163 rolls and counting, are perfectly fair, so I assume that what I've done has fucked with the underlying maths of the Chi-Square test in a way that needs to be accounted for.

Does anybody know the best way to approach this? I'm really enjoying the challenge of trying to quantify if a die has a bias towards a certain physical direction, but I've reached the limits of my statistical abilities and need some assistance.

Hi everyone,

I am considering applying to the MSc Statistics at the University of Geneva (UNIGE) for September 2027, and I would be very interested in hearing from current or former students, especially people who entered the programme with a background in business, economics, management or business analytics.

My academic background is:

• Bachelor in Management – HEC Lausanne (UNIL)
• Master in Management, specialization in Business Analytics – HEC Lausanne (UNIL)
• Several years of professional experience (5-6) mostly in HR since then.

I would now like to deepen my education in statistics and move towards a more quantitative, mathematical and analytical career. I am particularly interested in statistics, mathematical modelling and quantitative methods, rather than a programme with a strong focus on programming.

1. Admission

My Bachelor included the following courses in mathematics, statistics, computer science, quantitative methods, economics and finance:

• Mathematics I
• Mathematics II
• Statistics I
• Statistics II
• Statistics and Econometrics I
• Introduction to Logic
• Computer Models
• Programming
• Information Systems
• Business Intelligence and Analytics
• Risk Management
• Decision Analysis
• Operations Management I
• Financial Markets
• Principles of Finance
• Economics I
• Economics II
• Microeconomic Analysis
• Macroeconomic Analysis
• Corporate Finance
• Entrepreneurial Finance and the New Venture Funding Process

For those familiar with the programme: does this Bachelor background seem reasonably compatible with admission to the MSc Statistics?

In particular, I would be interested in hearing from people who were admitted with a business/economics/management background rather than a traditional mathematics or statistics degree.

Based on this kind of background, were significant prerequisite courses required? How important are previous courses in calculus, linear algebra, probability and mathematical statistics for admission and for successfully starting the programme?

2. Equivalences from my previous Master's degree

I understand that up to 30 ECTS can potentially be recognized as equivalences in the MSc Statistics, although the official decision is made after admission.

My Master in Management with a specialization in Business Analytics included the following courses:

• Machine Learning in Business Analytics (6 ECTS)
• Optimization Methods in Management Science (6 ECTS)
• Quantitative Methods for Management (6 ECTS)
• Data Science in Business Analytics (6 ECTS)
• Business Intelligence and Analyzing Big Data (6 ECTS)
• Programming Tools in Data Science (6 ECTS)
• Company Project in Business Analytics (6 ECTS)
• Conceptual Modelling in Business Analytics (6 ECTS)
• Strategic Modelling (6 ECTS)

Has anyone here entered the MSc Statistics with a previous Master's degree and successfully obtained substantial equivalences?

More specifically, how realistic would it be for some of these courses to count towards the 30 ECTS, depending on their content?

If you have been in a similar situation, how strict was the equivalence process in practice? Were previous Master's courses generally recognized when their content overlapped with the MSc, or was the process quite restrictive?

3. CCS / additional preparation

I am also considering taking some courses from the UNIGE Complementary certificate in applied statistics (CCS) before potentially starting the MSc in September 2027.

The idea would mainly be to strengthen my statistical background if my previous Master's courses are not sufficient to reach the 30 ECTS of potential equivalences.

For anyone familiar with the CCS and/or MSc Statistics:

Would this be useful preparation for someone with my background?

Are there particular CCS courses you would recommend to strengthen my foundations in statistics, probability or mathematical methods before starting the MSc?

4. Experience of the MSc and career outcomes

Finally, I would really appreciate feedback from current or former students about the programme itself :

• How mathematical/theoretical is the MSc in practice?
• How much programming is involved? Which of R and Python is used extensively? Both? I am comfortable with some programming, but I would prefer a programme where mathematics and statistics are more central than programming.
• How difficult was the transition for students coming from a business/analytics background?
• What types of jobs did you or your classmates get after graduating?
• Did you find the degree useful for finding a job in Switzerland?
• In particular, are there good opportunities in Lausanne or elsewhere in Switzerland?

I am especially interested in hearing from people who actually completed the MSc, particularly those who came from a non-traditional statistics background.

5. Alternative Master's programmes

Given my background and my goal of moving towards a more quantitative/statistical/mathematics career, would you recommend the MSc Statistics at UNIGE, or do you think there are other Master's programmes in Switzerland that might be a better fit for my profile?

If so, which programmes would you consider and why?

Thanks a lot to anyone willing to share their experience!

Hi everyone, I currently have a dataset where there’s 3 groups. Each group has 3 independent values, and I want to compare each group against each other.

As I’m a little concerned about the sample size, would it better to use:
1) Pairwise welch t-test with p-value correction, or
2) ANOVA with tukey test

Thank you in advance for the help!

Hi there, I'm currently working on my master thesis where I have a parallel mediation. I am working on the method section but i am so completely lost in how to check all the assumptions for the model.
From what I do understand I can do a visual inspection for the scatterplot (after using model 4) to inspect the linearity, homoscedasity and outliers.
The other 2 assumptions are normality and multicolinearity but I just don't understand how to do this. Could anyone help me? some explanation or links to proper resources would be greatly appreciated!

I'm about to start my final year as a Stats and Econ undergrad, I also plan to pursue Master's in statistics, hopefully in the same university as I am now, I checked and I can take a graduated level class this year and it will count to the necessary credits needed for the Masters program if i'll indeed continue in the the same uni

I've checked with 2 professors about their graduated level classes, they said that it seems that I have the necessary background for the their class, so i'm considering taking one of these 2 courses:

Casuel Inference - according to the syllabus it will cover: Causal Parameters, Randomization, confounding, selection bias, Usage of DAGs for checking assumptions and method and variable selections, ML algorithms in casual inference for the estimations of heterogeneous effects, Propensity score, matching, IPW and Instrumental variables.

Optimization under uncertainty - the syllabus doesn't really say as much but it says it will cover 3 main topics: Online approximation algorithms, Stochastic optimization and Onilne machine learning

So i'd love to hear some opinions on those subjects and which course sounds better to you strangers

Edit: The guy in the video mentions something about "iterative convergence". I'm assuming he means how the residuals and coefficients converge to their true values after multiple iterations

I recently started learning about Moving Average models and came across this video where the guy does an iterative process until which he gets the correct residuals and coefficients but I can't for the life of me understand the theory behind why it works.

Basically, for the very first iteration he assumes the errors are the demeaned values. He then regresses them against the Y variables and ends up with coefficients. He then calculates new residuals from the 1st iterative model and uses them as the regressors for the next iteration. He repeats this until the residuals and coefficients barely change.

Why and how does this work?

The only thing I'm familiar with for the MA residuals process is Maximum Likelihood but that's not what he's doing here at all.

Thank you very much

I'm a former math major who's now majoring in philosophy and doing a minor in statistics. I'm taking 3 statistics courses next semester and have room for one more math course. I'm debating between an intro to proofs course or an applied biomath course. Below are the course descriptions.

MATH 141W: Intro to Mathematical Proofs and Combinatorics

• An important part of mathematics research is investigating and discovering patterns, then proving the patterns hold in all cases. This course focuses on solidifying these important skills. By studying mathematical logic, methods of argument, and a wide variety of topics, students will better understand what it means to prove things and how to do so rigorously. By working to discover the underlying structure and truth in various mathematical topics, students can expect their problem-solving skills to improve, as well as their ability to form and communicate persuasive, logical arguments.
• Students planning to take MATH242: Introduction to Analysis I are strongly encouraged to take MATH141.

MATH 360: Intro to Biomathematics

• In this course you will learn core concepts and modelling techniques in mathematical biology across a range of topics, from competition and predator-prey interactions to infectious diseases and evolution. These topics will be taught in the context of real-world scenarios where mathematical models are used to inform decision-making, such as invasive species, pest control, and pandemics.
  
• Even relatively simple mathematical models can give powerful insights into many biological phenomena, from patterns of biodiversity to population crashes, and can help inform policymakers about how best to manage human interactions with the natural world. You will learn how to construct models, analyse them algebraically, geometrically, and numerically, and then interpret the results biologically.
  
• Examples of the types of questions we will answer include:
  • How do species respond to human-driven pressures such as climate change and overfishing?
  • When are competing species able to coexist?
  • How best to respond to an epidemic?
    
• Throughout the course, we will explore case studies in class where we work through longer problems together, learning not only how to analyse models, but also how to construct them wisely and communicate the results. In addition to developing core analytical and computational skills, you will gain transferable skills in critical thinking, problem solving, and communication.

Which one would you recommend I guys take? I'm someone who is extremely weak at proofs which was why I switched out of the math program to begin with. However, I know real analysis is an important part of statistics, so I figure the intro to proofs course will be of great assistance.

The biomathematics course sounds interesting and will probably be easier for me since I'm strong at computational math. However, the content looks like it will utilize a lot of differential equations and I'm unsure how practical this will be for statistics. Any advice is greatly appreciated!

I'm sharing this new tabular data diagnostic tool (Entropic Scree). It can be used to estimate these properties of your high-d, real-world, dirty dataset:

• The informational volume of the signal (i.e., helps you assess whether the signal is strong enough to survive the dataset's idiosyncratic volume).
• The overall signal-to-idiosyncratic volume ratio (SNR).
• The intrinsic rank.
• Provides an exploratory map that allows for the identification of decoupled sub-networks of variables.
• The linear sufficiency (i.e., does the dataset align with the linear assumptions of standard PCA?).

Instead of evaluating linear variance, rank order, or Euclidean distance like traditional PCA variants, this new method evaluates a transformed mutual information metric. Relative to these baselines, it is less reliant on strong parametric or distance assumptions, making it appropriate to apply more broadly.

It also serves as a practical diagnostic of the theory explored in the From Garbage to Gold framework, which describes when and why uncurated, error-prone data can be used directly to create accurate prediction models.

There is a preprint that presents the full technical details, and Python and R packages will be released soon. Right now though, the original function is already available in R (see Quick Start R Function Code below).

• Entropic Scree Preprint: https://doi.org/10.5281/zenodo.22028087
• Entropic Scree GitHub: https://github.com/tjleestjohn/entropic-scree
• From Garbage to Gold Preprint: https://arxiv.org/abs/2603.12288

Let me know how it goes if you give it a try... or if you have any questions or comments of course.

############ 
# Quick Start R Function Code.
# To load the function, copy and paste the following into your R console, then hit enter. 
############

# 1. Define the direct URL to the raw function script on GitHub
url <- "https://raw.githubusercontent.com/tjleestjohn/entropic-scree/main/Entropic.Scree.R%20-%20ENLI.R"

# 2. Define what you want to name the file on your computer
file_name <- "Entropic.Scree.R - ENLI.R"

# 3. Download the script to your current working directory
download.file(url, destfile = file_name)

# 4. Source the core function into your R environment
source(file_name)

# 5. Ex. To run the function and extract bipolar modules:
# results <- Entropic.Scree(dt 
#                         , extract_bipolar_modules = TRUE)
#
# View the extracted structural sub-networks for the primary axes:
# results$bipolar_modules

My brain revolts when I’m trying to learn advanced math, even though I generally have no problem with abstract concepts. Please help.

Edit to add: thank you so much for all the great explanations!!!! I cannot reply to them all, but y’all have helped me greatly with actually understanding these data sets in my homework, and I am grateful 🩷

Hi there,

I've created a video here where I explain how generalized linear models work.

I hope some of you find it useful and as always, feedback is very welcome! :)

Hello,

I have a question about LMMs:

Repeated-measures design, 2x2:
One continuous dependent variable
Two independent variables, each with two levels

I want to test for interaction; the hypothesis is directional.

First, a linear mixed model was estimated in R wirh lme:
AV ~ UV1 * UV2 + (1l Participant)

Then a Type 3 ANOVA with lmertest

And finally, using emmeans, simple contrasts and a planned directed interaction contrast

My question: In a 2x2 design, is it even necessary to run a Type 3 ANOVA after the LMM?

And is the planned directed interaction contrast even necessary?

I’m also confused by the fact that some people report LMM results in the text using b, while others use F.

I’d appreciate any answers.

Hi everyone! I’m a beginner in Data Science and learn best by building projects.

I’m looking for GitHub repos, YouTube playlists, or websites with project-based learning from beginner to advanced level, ideally covering different ML models such as regression, classification, clustering, tree-based models, boosting, NLP, time series, etc.

I’d also love end-to-end projects that include data cleaning, EDA, feature engineering, model building, evaluation, and deployment.

Any recommendations?

Hello! I’m a post-Bacc student (Business Background) interested in applying for Statistics, Applied Stats and Data Science Masters programs this Fall.

I’m primarily interested in Time Series/Bayesian/Causal Inference. I have finished prerequisites in Calc 1-3, Probability, Linear Algebra, Python/SQL

Ideally, I want to work in Environmental, Tech, and Healthcare industries, but given the market- I really just want a flexible degree that employers can find value in.

I’m looking for advice and insight into how a Masters experience in this field looks like such as:

- What did you think your degree did wrong/right?

- What is worth prioritizing in this field? (Ex. Courses/Projects/Networking/Coding/Theory etc.)

- Is brand name a big factor?

- Does online/in-person/out of state make a difference?

- What career prospects (Scaling pipelines vs. Decision Scientist vs. Data Cleaning/Visualization) to realistically expect and prepare for in this market with a Masters degree?

Uni student here trying to do some projects to put on my resume. I made a logistic model of hospital readmissions and the model doesn’t do that great. AUC is about .64. For what it’s worth, the National Library of Medicine was only able to get .67 for their model. Would I be able to reference that in my defense? But my question is if it’s worth putting my project on my resume? Or is it just going to show I don’t have the skill to make a predictive model?

The whole project consisted of data cleaning and ofc making the model and model comparisons such as AIC. Cause of imbalance (1:9), I also lowered the threshold for positives (from >.5 to >.15 as positive)

Hello. I’m not sure if this is the right forum to post but I need some help. My father is a retired statistician. He was an actuary. He did some important work in “stochastic models for prostate specific antigen levels” and some “early predictive modelling relevant to the insurance industry for HIV/AIDS” - I don’t really know much about this stuff other than what google tells me.

Anyway, I would like to buy him a watch and I’m looking for something to represent what he did and I’m afraid his name is too bloody long to fit!

The 2 leading candidates for symbols, I think, are 1. The Kolmogorov forward operator (Pij(t)) and 2. The actuarial halo - the present value annum. Which looks like an “a” with a horizontal and vertical slash with a “n” inside it.

Can anyone help? Are these appropriate? Is there any better?

I don’t want to get this wrong.

Thank you.

Hi all! I am wondering what is your view on causal inference methodologies such as g-computation, iptw, ps matching, marignal structural models etc. Do you think they should be used only in an causal framework accompanied by DAGs, and proper causal language?
Would you consider appropriate if they were used for more exploratory work that does not claim causality?
I may not be communicating my question very well so here are some exmaples:

1) Binary logistic regression: In the biomedical field it is extremely common that standard observational and/or exploratory studies use logistic regression for all inferences with odds ratios being the main reported result. I don't see why someone couldn't use marignal standardization using the same logistic regression model in order to calculate a marginal absolute risk and/or risk differnece for the exposure of interest. I am wondering why this is not common.

2) Propensity score based methods: Causal inference operates under very strict and usually difficult to verify assumptions. When examining the effect of an intervention on an outcome and assuming that some of the assumptions for causal inference are violated (e.g. unmeasured confounding), would you prefer a paper that still uses PS-based methods but refrains from using causal language, or a paper that uses more standard methods such as regression and sticks to associations and exploratory framing?

In short do you think these methods should be used only under the causal inference framework making sure that all assumptions are true and a well-thought DAG is provided, or do you see them as methods that can be used for associations as well in order to reduce at least some of the bias introduced by other methodologies?

Hello all,

I’m looking to improve my foundational understanding of statistics and work my way through to advanced methods by reading textbooks.

Could you please suggest the best text resources for these purposes, ideally ranked/listed in the order I should read them?

For context, I’ve worked as a data analyst and quantitative researcher for 5 years, but I’ve always felt slightly weak in my understanding of stats so now I want to just start from zero to build a super solid foundation.

Do I need a computation if I use convenience sampling in my data analysis? (I don't know if this is the right community to ask this question, I hope you can be kind)