It matters as a common standard for model integration. System messages aren't removed when regular context messages start to get cut, the model pays more attention to tools and other directives defined there. Like situational context, language it should use, type of responses, etc. OAI/Anthropic give their models system prompts a mile long to tune their behaviour to a T.
There's also the ideological camp with Hartford and Nous and the rest where models are supposed to be trained as generally as possible, with the system prompt being strictly followed to adapt it to specific use cases.
