The exclamation mark is thought to have developed from a Latin expression, ‘io’, used to express joy or triumph. Sometime around the early 15th century, scribes began to condense ‘io’ into a single space, with the I placed above the O, eventually forming a single character, ! (via @HaggardHawks)