Crystal structure prediction (CSP) is a foundational component of high-throughput materials discovery and design. Despite the success of classical CSP methods, their applicability to large, complex systems of physical interest has been limited by the computational cost of methodically exploring an exponentially large configuration space. Additionally, the underlying optimization algorithms require an initial guess structure and are prone to becoming trapped in local minima. Recently, artificial intelligence models, such as transformer-based large language models (LLMs), have shown promise in learning crystal design rules directly from structural data. A wide range of crystal representations have been used for this task; however, not all treat key physical constraints, such as symmetry, on equal footing. In this work, we evaluate the impact of explicitly encoding symmetry constraints in the textual crystal representation on the performance of an LLM fine-tuned for crystal structure prediction. Initial model examination indicates that models with explicit symmetry encoding generate valid structures at a slightly improved rate compared to general implicit symmetry models and have comparable accuracy relative to ground truth crystal structures.