Geoclidean: Few-Shot Generalization
in Euclidean Geometry

There exist numerous systems of geometry, each with its own logical system. Each logic can be described in a formal language, each formal language describes construction rules, and each set of construction rules describe concepts in the geometric universe. Geometric concepts can be realized and rendered into images. Euclidean geometry was among the earliest formalized, first described by Euclid in Elements (Simson et al., 1838). The first book of Elements details plane geometry, laying the foundation for basic properties and propositions of geometric objects. Importantly, Euclid’s geometry is constructive—objects only exist if they can be created with a compass and straightedge. Euclidean constructions are abstract models of geometric objects, specified without the use of coordinates and without concrete realizations into images. Hence, changes to the image rendering, such as alterations to size and rotation, do not change an object’s intrinsic Euclidean geometry.

Euclid’s axioms build the foundations of this geometric universe via 1) draw a straight line from any point to any point, 2) describe a circle with any centre and distance. The Euclidean geometry universe is determined by these base geometric primitives, and the constraints that parameterize object relationships. There are other universes that contain different logic systems, from Descartes’ analytic geometry, to Euler’s affine geometry, to Einstein’s special relativity. Because of its simplicity and abstraction we are particularly interested in the universe of Euclidean geometry – do humans and machines find these constructions natural? We later explore whether they spontaneously generalize image classes according to Euclidean rules.

We create a domain-specific language (DSL), Geoclidean, defining Euclidean constructions that arise from the mathematics of Euclidean geometry. Geoclidean allows us to define construction rules for objects and their relations to each other, encompassing concepts in the Euclidean geometry universe. It includes three simple primitives. The first is a point, parameterized by constraints, if any, to an object or a set of objects previously defined. The point is defined without specific coordinates, and only when realized, would be assigned coordinate values $x$ and $y$. The second is a line, which, following the first axiom, is parameterized by two points, representing the beginning and end. The third is a circle which, following the second axiom, is defined by a center point and an edge point. These primitives represent the compass and straightedge constructions that Euclid introduced. Lines and circles are defined by points, while points can be constrained to previously built objects. In this way primitives are sequentially defined to form a concept.

Geoclidean’s syntax is defined in Table 1. We can initialize a new point, line, or circle via the constructors Point, Line, Circle, assigning them to a named variable. For points we generally use a shorthand to define and use the variable inline. For example, p1() is a free point that is unconstrained to any objects, and can be realized anywhere in the image, p2(circle1) is a partially constrained point that lies on the object circle1, and p3(line1, line2) is a fully constrained point that lives in the intersection of line1 and line2. A point can be reused by referring to its name. The line Line(p1, p2) is parameterized by two end points, and the circle Circle(p1, p2) is parameterized by points at the center and edge. Not indicated in Table 1 is the semantic constraint that variable names must be defined before being used within constructors. Visibility of rendering is denoted for each object, with * indicating that the object is not visible in the final rendering, as some objects in Euclidean geometry are used solely as helper constructions for other core objects in the concept. Each construction rule is a geometric object, and a sequence of construction rules define a Euclidean geometry concept.

Geoclidean implements this Euclidean language and renders realizations of concepts from the language into images. For each line or circle object, we realize its rendering based on the point parameters required. During rendering, each point parameter is given randomly sampled real-valued coordinates bound by its constraints. If the point exists already, Geoclidean reuses the past component; if it is a free point without constraints, Geoclidean randomly samples values for $x$ and $y$; if constrained, Geoclidean randomly samples a point that lies constrained on the object or the intersection of a set of objects. It is possible for intersection sets to be empty, and we reject sample realizations until finding a satisfying realization for all points. Geoclidean creates the objects sequentially and renders them into an image, if visible.

We see in Figure 2 an example of the Geoclidean language describing the equilateral triangle concept realized into an image. The construction rules in Algorithm 1 are created step by step; we color each object for clarity. The first construction rule creates a red line between two unconstrained free points, p1() and p2(); when realized with sampled real-valued points, this line can be anywhere in the image with any length and rotation. The second rule creates an invisible orange circle with the ends of the first line as its center and edge point. The third rule creates another invisible yellow circle with the center and edge point flipped, forming intersecting helper circles with the same radius. Then, the fourth rule creates a green line between p1 and a new point p3(c1, c2), which is a point constrained to the intersection of the previously created orange and yellow circles (the realization randomly chooses one of the two intersection points). The last rule creates the final blue line between p2 and p3, completing the constructed triangle and enforcing all sides to be of equal length. This concept is that of the equilateral triangle, and we see that the invisible helper circle objects serve as essential constraints to the final rendering.

Importantly, the construction rules do not specify any coordinates, and our Geoclidean framework creates the coordinates upon realization of the concept into an image. Hence, the rendered equilateral triangle can be of any size and orientation, while always respecting the underlying geometric concept. In Geoclidean, every random realization of this concept creates equilateral triangles, as represented in Euclid’s geometric universe. The Geoclidean framework allows us to create rendered image datasets that follow the specified concept language.

We collected human judgements for the Geoclidean few-shot concept learning task. We recruited $30$ participants for each concept using the Prolific crowd-sourcing platform (Palan and Schitter, 2018). As mentioned above, participants are given five example realizations of the target concept and $15$ test images including five positive examples, five Close negative examples, and five Far negative examples. Each of the test questions states: “Each of the five images shown is a ‘wug’. Is the image below a ‘wug’ or not a ‘wug’?”, where ‘wug’ is a random made-up word for each task. The order of tasks and realizations is randomized for each participant to remove order effects. The experiment interface was implemented on Qualtrics, with details described in the Appendix.

We report task accuracy in Table 2, scored as the percentage of participants correctly categorizing test images, averaged across all examples. We split the $15$ test images into two tasks, with the Close task consisting of $5$ positive examples and $5$ negative examples from Close, and the Far task consisting of the same $5$ positive images with $5$ negative examples from Far. Each task contains $10$ examples in total. We see that human performance is strong across all tasks, with on-average higher scores in Far compared to Close, showing that the number of differences in construction rules affects the semantic distance between rendered realizations. Only tasks LLL, CLCL, and rhomboid yielded slightly better performance in Close than Far. Out of all tasks, LLC, CCC, LLLC with Close negatives are more difficult for humans (with equally poor performance across all test images), which we hypothesize is due to more subtle constraint intersections. In general, humans perform well on this generalization task.

We show accuracy histograms in Figure 4, with the left two plots depicting results from concepts in Geoclidean-Elements and Geoclidean-Constraints, and the right two plots depicting results when calculated with Close negative examples and Far negative examples. Humans are more sensitive to concepts in Geoclidean-Elements, which are complex constructions that test the exactness of shapes (e.g., squares and equilateral triangles), and slightly less sensitive to concepts in Geoclidean-Constraints, which test the precise relationships between objects (e.g., constrained contact point between the end of a line and the center of a circle).

Participants could generate infinitely many rules consistent with the positive images seen in the few-shot examples (e.g., a “wug” can have its own prototype for each example, as there are no negative examples), and there is potential ambiguity as to which are the correct construction rules of the concept. Despite this wide range of possible generalization patterns, the generalization rule chosen by humans corresponds well to the Euclidean construction universe.

We benchmark pretrained vision models’ performance on the Geoclidean task, to evaluate few-shot generalization (with no meta-learning or fine-tuning) in the Euclidean geometry universe. We measure performance of features from ImageNet-pretrained VGG16 (Simonyan and Zisserman, 2014), ResNet50 (He et al., 2016), InceptionV3 (Szegedy et al., 2016), and Vision Transformer (Dosovitskiy et al., 2020), and evaluate both low-level features and high-level features for each of the models. Low-level features are outputs of earlier layers in neural networks, which tend to capture low-level information such as edges and primitive shapes, while high-level features are outputs of later layers that tend to capture more high-level semantic information (Zeiler and Fergus, 2014). We detail how we define the layers for each baseline in the Appendix. To evaluate these features, we extract features $\phi$ from the few-shot $t$ reference images of the target concept, to create a prototype target feature, $T=\frac{1}{n}\sum_{i=1}^{n}\phi(t_{i})$. We classify a test image $r$ as in-concept if it is closer than a threshold to the prototype: $f|T-\phi(r)|<\theta$, where $f$ is the normalizing function between $0\sim 1$. The threshold is fit by selecting the best-performing normalized distance threshold across all 74 tasks, for given features. (By fitting this free threshold, we bias the reported accuracy in favor of models.)

In Table 3, we present accuracy of ImageNet-pretrained low-level and high-level features across different models. Humans substantially outperform features from vision models, showcasing the gap between human and model capabilities in Euclidean geometry concept learning.

We see that, on average across tasks, pretrained vision models perform poorly compared to humans. In general, high-level features perform slightly better than low-level features, though there are cases where this differs. Interestingly, the high-level features from the Vision Transformer (ViT) outperform its convolutional network counterparts, and achieve human-level performance in some tasks. There are a few tasks where the visual feature accuracy outperforms humans, notably tasks CCC, LLLC, CCCC, CCT, where high-level features from ViT perform strongly. We hypothesize that this is because humans are not as sensitive to details of intersections between circles.

In Figure 5, we present histograms comparing maximum low and high-level feature accuracy to human accuracy, illustrating the gap in performance. In Table 4, we report the Pearson correlation coefficient between the answers of humans and models. We see that humans are generally more aligned with high-level features than low-level ones. Additionally, though ViT achieves high accuracy, it is not correlated to human performance, indicating failure to generalize in the most human-like way.

We report additional comparisons of low-level and high-level visual features from ImageNet-pretrained VGG16, ResNet50, InceptionV3, and Vision Transformer in the Appendix, and show that similar trends follow across data splits and performance metrics. We also include low-level and high-level feature visualizations in the Appendix, comparing Geoclidean tasks that require reasoning, to perception tasks involving simple geometric primitives and perturbations. These comparisons highlight Geoclidean as a unique and interesting test for vision models.