There are more than 3.600 archives in Germany - big ones, small ones, and very small ones. In the section "Discover German Archives" you can find lists of the more important ones that also have a webpage. This should get you started, but for highly specialised research it might very well be necessary to consult tiny private archives. The regional web portals of the German states are a great help for finding these archives. Focussing on specific regions you can search for relevant archives.
Furthermore, there are meta search engines helping you locate relevant material in different archives.These are particularly useful if you are looking for the papers of a historical figure, their nachlass. Since public archives are not responsible for a private nachlass of for instance an artist or a scientist – unless they have been of national importance – it is often difficult to find. Nachlasse, however, are not only held by archives but can also, quite frequently actually, be found in the collections of libraries.
One example: You are doing research on Otto von Bismarck and are therefore looking for his nachlass. This is rather easy using the Zentrale Datenbank Nachlässe. Here you will find out that his private papers have been divided: A part of them is held by the family-archive of the von Bismarcks in the little village of Friedrichsruh. Another part is held by the Bundesarchiv in Koblenz. The next step is to consult Kalliope, a database of autographs. This search engine will help you to identify particular documents, for instance individual letters to or from Otto von Bismarck that can be found in other people’s papers or collections. All in all, Kalliope lists 260 documents to or from Otto von Bismarck that are held by various archives or libraries in the whole of Germany.
But beware: Far from all nachlasse are included in these databases. Hence, do not give up if your search does not return any useful results. Consider where your person of interest was born, where he or she was working, where he or she died. Maybe the nachlass is held by the town-archive of the birthplace? Maybe the company he or she worked for has some records? Maybe your person of interest was of regional importance and there is a foundation preserving his or her memory? It is best to look for possible institutions using the regional archival web portals of the German states, listed in the left box.