In this study, varying concentrations of polyunsaturated fatty acids in goat’s milk were obtained by supplementing goat feed rations with maize dried distillers grains with solubles (DDGS). In comparison to the control group, the milk of goats that received DDGS contained about 25% more polyunsaturated fatty acids, 30% more conjugated linoleic acid (CLA), and had more favorable values of the atherogenic and thrombogenic indices. The diverse composition of fatty acids present in the goat’s milk correlated with the olfactory perception of the milk and its volatile compound profile. The level of unsaturated fatty acids significantly affected the odors detected in the processed goat’s milk. The volatile compound profile of the milk was mostly differentiated by the concentrations of aldehydes (hexanal and furfural) and furanones (furaneol), which exceeded their odor threshold values. The total concentration of identified aldehydes was highest in pasteurized milk samples from the goats of the experimental group, as a result of heating. Furfural was found in pasteurized samples of both the control and experimental milk at concentrations exceeding its odor threshold (3 mg·kg−1). Feeding goats with DDGS did not change the animal, cooked, dairy fat, lack of freshness, light-oxidized, or overall dairy flavors of the raw milk. After pasteurization, milk from goats fed with DDGS retained its characteristic animal odor and gained a very intense cooked flavor.