As much as apples, cars, are not part of mathematics, in the same way is not rectangle or any other geometrical figure. Geometry is perhaps interesting because it selects, generates certain sets of numbers that are of interest in everyday applications, like lines, squares, rectangles, circles. It is quantification of these figures and their measures that matter to mathematics, and not figures themselves. The thought process that takes place in defining a circle as an ideal abstraction of all real world attempts to make a circle (as well as a straight line abstraction of all straight directions) is a nice thing to think about, but that’s not part of mathematics. Once you “idealize” circle, math cares only about the numbers you provide by measuring them.
Simply put, no geometric figure should be considered an element or part of pure mathematics because none of the theorems in math are proven using them. If seemingly geometry terms are used in proofs or appear to be a focus of study, like trigonometry or differential geometry, it is because the axioms of geometry are part of it, but, they are not part of mathematics. Mixing ZFC axioms and geometry axioms is like mixing ZFC axioms and axioms of any other system, including "marbles used in counting", carrots methods of purchase, quantitative finance rules, etc..
"The formalist makes a distinction between geometry as a deductive structure
and geometry as a descriptive science. Only the first is mathematical. The use of
pictures or diagrams or mental imagery is nonmathematical. In principle, they
are unnecessary. He may even regard them as inappropriate in a mathematics
text or a mathematics class." ("What is Mathematics Really" Rueben Hersh)
Of course, it doesn't mean you should not use them to better communicate your ideas, investigate new directions in math or other sciences, or visualize a bit more difficult concepts in mathematics.But, you have to clearly differentiate between mathematics and these non mathematical objects and concepts.
Mathematics is defined not by objects it counts, nor by reasons or logic why those objects are counted, but with concepts used to define mathematics axioms and to define proofs of mathematical theorems.