Inductive and Selection Bias

#02 Data analytics

Hello!!

I hope you enjoyed our previous newsletter on how machine learning is shaping the future of predictive data analytics. Let’s continue to our previous discussion i.e inductive bias

There are two types of inductive bias that machine learning algorithm can use:

•Restriction bias

•Preference bias

•1. Restriction Bias

•This type of bias limits the set of models or hypotheses that the learning algorithm can consider. In other words, it imposes constraints on what kind of patterns or functions the model can learn. It narrows down the model's options from the beginning.

•Example: If you use a linear regression model, the restriction bias is that the model can only learn linear relationships (straight lines) between variables. No matter how complex the data is, the model will only try to fit a straight line.

•Why It's Useful: By limiting what a model can do, it can be simpler and faster to train, and it might avoid overfitting complex data.

•2. Preference Bias

•This bias doesn't limit what the model can learn, but instead prioritizes some models or solutions over others. It is about giving preference to certain types of patterns or solutions based on the algorithm's design, even if it could technically learn other patterns.

•Example: In a decision tree, you might have a preference bias toward shorter trees (with fewer splits) because they are simpler and more likely to generalize well, even if deeper trees could fit the training data perfectly.

•Why It's Useful: Preference bias guides the learning process to favor solutions that are likely to generalize well on unseen data, balancing complexity and accuracy.

Machine learning algorithms use two sources of information to guide the search, the training dataset and the inductive bias assumed by algorithm.

•1. Sampling Bias

•Sampling bias happens when the sample (or subset of data) used to train a model is not representative of the overall population. This leads to a dataset that doesn't accurately reflect the diversity or true characteristics of the target population.

•Cause: Sampling bias can occur when certain groups are overrepresented or underrepresented in the dataset. It often happens if the data is collected from a biased source or if a non-random sampling method is used.

•Example: If a survey on internet habits is conducted only among people in urban areas, it might miss insights from rural areas, leading to a biased conclusion that may not apply to the entire population.

•Impact: A model trained on a biased sample may make incorrect predictions when applied to a broader, more diverse population.

•2. Selection Bias

•Selection bias occurs when the way participants or data are selected for the study causes certain groups or characteristics to be excluded or favored, leading to biased results. It's a broader concept that includes sampling bias as a specific type.

•Imagine you're trying to figure out the favorite ice cream flavor of all students in a school, but you only ask people in the cafeteria during lunch. You're likely missing students who bring lunch from home, eat somewhere else, or skip lunch. Your results might show that "cafeteria ice cream" flavors are the favorites, but that doesn't really represent the whole school—it only tells you about the people who were in the cafeteria.

•Impact: Selection bias can make the results of a study misleading because they do not accurately represent the population that the model or research is intended to cover.

Key Difference

•Sampling Bias is specifically about the sample not representing the entire population, often due to how the data is gathered.

•Selection Bias is a broader concept involving any systematic error in the selection of participants or data points, which can lead to certain groups being excluded or favored.

•In summary:

•Sampling bias is about who ends up in the dataset.

•Selection bias is about how the dataset is put together including what might have been excluded, intentionally or unintentionally.

•Consequently, data analysts need to think about the sources of the data they are using and understand how the data was collected and whether the collection processes introduced a bias relative to the population. They also need to reflect on the processes they use to preprocess and manage the data, and whether any of these processes introduce bias into the sample .So, in summary, although inductive bias is necessary for machine learning, and in a sense, a key goal of a data analyst is to find the correct inductive bias, sample bias is something that a data analyst should proactively work hard to remove from the data used in any data analytics project.

Thank you for joining us! if you enjoyed this edition, consider giving it a like. We’d love to hear your thoughts-drop a comment below!