What are some great open source Java libraries that every java developer should know?

Dependency-injection/inversion-of-control frameworks. This is such a powerful tool for organizing large software systems that, when I first learned about it, it was like a blast of dynamite had cleared out my head.  The canonical example is the Spring framework, but Google has its Guice framework as well.

Web application frameworks. Like it or not, one thing Java is really good at is developing Web applications, so you may be doing this a lot.  There are many Web frameworks out there that can make your life a lot easier than just using raw Java Servlets and JSPs.  Spring has one; there are also Struts, JSF, and Tapestry out there, among many, many others.  For a change of perspective on the way Web applications are written, look at Apache Wicket; it'll bend your brain if you're used to the standard paradigm of Web applications, but it's very powerful.

Object-relational frameworks. Likewise, your Java code is likely to be storing and retrieving information from a database, and while it is possible to work with raw SQL from Java, it's emphatically not fun.  Hibernate, EclipseLink (ex-Oracle Toplink), and MyBatis (ex-iBATIS) all solve the issue of bridging between Java objects and SQL tables, in different fashions.

Two points I'll leave you with:

1. There are many other choices for each of these than I've outlined here.  Have a look around.  If nothing else, the more familiar you get with frameworks of these types, the better you can judge between them.
2. There is no "one right answer" for any situation.  There are various choices you can make, and, in fact, one of the choices you may make is "roll your own" if none of the frameworks out there suits your needs.  But the best way to make that choice in an informed manner is to become familiar with the existing tools for the job, and to know their strengths and weaknesses.