The First Amendment to the Constitution says: "Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the government for a redress of grievances."
In the 45 words of this amendment do you see a so-called "establishment clause" ? a phrase that allegedly bans the display of religious symbols on public property? Me, neither.
Actually do. Just do a search for the word "establishment". It seems fairly clear. Congress cannot pass laws respecting an establishment of religion. It's fairly minor judicial activism to read that as prohibiting judges from displaying religious icons as they enforce the laws.