Weights

So, we know how data flows into our simple neuron, but how do we transform this data into actionable knowledge? How do we build a model that takes these input features, and represents them in a manner that helps us predict the weather on any given day?

Giving us two features that we can use as input in our model, for the binary classification task of determining rainy or sunny days.

Well, the first step will be to pair up each input feature with a respective weight. You can think of this weight as the relative importance this specific input feature has with respect to the output class that we are trying to predict. In other words, the weight for our input feature temperature should ideally reflect exactly how much this input feature is related to the output classes. These weights are randomly initialized at first, and are learned as our models see more and more data. Our hope here in doing this is that after enough iterations through our data, these weights will be nudged in the right direction, and learn the ideal configuration of temperature and pressure values that correspond to rainy and sunny days. We actually know, from domain knowledge, that temperature is highly correlated to weather, and hence will expect our model to, ideally, learn heavier weights for this feature, as data propagates through it. This can be somewhat comparable to the myelin sheath that covers axons in a biological neuron. If a specific neuron fires frequently, its myelin sheath is said to thicken, which insulates the axon, and allows the neuron to communicate faster next time.