STUDY / 02

Explore Classification

Learn boundaries between labeled classes.

Decision tree

1 tree

Cyan circle: class 0Coral square: class 1Darker shading: stronger prediction

99%training accuracy

95%test accuracy

0.065test log loss

Inspect the tree

y < 0.35?
Yes:
  Leaf: 100% coral (22 points)
No:
  y < 0.63?
  Yes:
    x < 0.27?
    Yes:
      Leaf: 100% coral (10 points)
    No:
      … more splits
  No:
    y < 0.67?
    Yes:
      Leaf: 67% coral (3 points)
    No:
      Leaf: 100% coral (19 points)

Each split chooses an axis and threshold to reduce class mixing.

Inspect held-out predictions

Test point	x	y	Actual class	P(coral)	Correct?
1	0.414	0.409	Cyan circle	0.000	Yes
2	0.499	0.150	Coral square	1.000	Yes
3	0.410	0.582	Cyan circle	0.000	Yes
4	0.625	0.190	Coral square	1.000	Yes
5	0.363	0.440	Cyan circle	0.000	Yes
6	0.716	0.771	Coral square	1.000	Yes
7	0.638	0.557	Cyan circle	0.000	Yes
8	0.158	0.468	Coral square	1.000	Yes
9	0.606	0.379	Cyan circle	0.000	Yes
10	0.409	0.812	Coral square	1.000	Yes
11	0.639	0.484	Cyan circle	0.000	Yes
12	0.160	0.427	Coral square	1.000	Yes
13	0.354	0.472	Cyan circle	0.000	Yes
14	0.801	0.649	Coral square	0.667	Yes
15	0.551	0.617	Cyan circle	0.000	Yes
16	0.806	0.316	Coral square	1.000	Yes
17	0.451	0.619	Cyan circle	0.000	Yes
18	0.694	0.793	Coral square	1.000	Yes
19	0.549	0.385	Cyan circle	0.000	Yes
20	0.587	0.146	Coral square	1.000	Yes
21	0.464	0.621	Cyan circle	0.000	Yes
22	0.643	0.789	Coral square	1.000	Yes
23	0.595	0.576	Cyan circle	0.000	Yes
24	0.574	0.162	Coral square	1.000	Yes
25	0.614	0.472	Cyan circle	0.000	Yes
26	0.750	0.260	Coral square	1.000	Yes
27	0.593	0.442	Cyan circle	0.000	Yes
28	0.828	0.339	Coral square	1.000	Yes
29	0.515	0.644	Cyan circle	0.667	No
30	0.278	0.220	Coral square	1.000	Yes
31	0.630	0.583	Cyan circle	0.000	Yes
32	0.862	0.543	Coral square	1.000	Yes
33	0.354	0.460	Cyan circle	0.000	Yes
34	0.654	0.212	Coral square	1.000	Yes
35	0.379	0.440	Cyan circle	0.000	Yes
36	0.697	0.221	Coral square	1.000	Yes
37	0.462	0.636	Cyan circle	0.667	No
38	0.785	0.330	Coral square	1.000	Yes
39	0.388	0.589	Cyan circle	0.000	Yes
40	0.373	0.836	Coral square	1.000	Yes

A perfect fit can be a warning. ↗

Compare linear SVM and RBF SVM on rings: a straight boundary cannot separate the inner and outer classes. The RBF kernel uses exp(−γ‖x−x′‖²). SVMs use a soft-margin C-SVC objective and a bounded SMO solver; margin contours are sampled on a display grid. For kNN, compare k = 1 and k = 15 with noisy labels. Logistic regression minimizes mean cross-entropy plus λ‖w‖²/2. All models use the same training split. SVM reference ↗ Add 20% label noise, then increase tree depth. Training accuracy can climb while test accuracy stays flat or falls. Compare the forest. For the neural network, start with two hidden neurons, then try eight and train again. This is a small, single-hidden-layer teaching model. Explore the full TensorFlow Playground ↗

Source / Andrej Karpathy ↗