Strictly, expert systems need not be rule-based. Could store knowledge as (e.g.) frames instead.
The term has no exact definition, but is, like ``database'' or ``word-processor'', a vague one, often misused by salesmen. It would have been generally agreed in the 80s, however, that certain properties distinguish an expert system from other software:
But here, the word ``symbolic'' comes from the same place as in ``symbolic AI''. It refers back to the symbolic AI tradition of knowledge representation, and implies that the knowledge which causes the system to make its decisions (the system's expertise) is coded in a form which can be made explicit to the human user. This allows him or her to appreciate how and why the decisions were made.
I have started to see ``neural net'' expert systems appearing. Presumably, these can't explain their conclusions, so maybe the sense of the word is changing slightly.
Other words for expert system: intelligent knowledge-based system (IKBS). I think this one was popularised by the British government in 5th generation project.
In the early days, ``expert system'' and ``production system'' were synonymous. Nowadays in AI, production system almost always means a rule-based cognitive model. These always restrict the style of inference and STM to reflect assumptions about the mind's cognitive architecture.