Nope, while you are right about the constant conflict with the Wehrmacht (which was very much intended by Hitler), the SS had designated military structures (first called Verfügungstruppe,then Totenkopfverbände,later the infamous Waffen-SS) that were acting as regular military formations and for quite a while were subordinated under the Wehrmacht command.

These military parts were having a fully military structure (that later on was kept within the SS but directly under Hitler’s high command nevertheless), service within these SS formations did “count” the same as regular service, people could get drafted into these formations instead of the Wehrmacht(especially when they were Germans who lived in a conquered country as these could not be part of the Wehrmacht). They had tanks, etc.

So from a purely technical standpoint they should probably be more seen as a separate,even more evil,military branch - none would say the US marines are not part of the military because they are not army, neither would someone say something like that to the Carabinieri.

BUUUUTTTTT: The SS also had the “regular” non military SS (Allgemeine SS) which was more based on a executive branch and absorbed the German police forces, customs, etc. in it, but also worked in various other functions, from guarding concentration camps and “behind the front lines work” to wartime economic stuff. (So in terms of crime not even a bit less evil than the Waffen-SS) Von Braun was a (very keen and voluntary) member of this regular SS.

So… Yeah… If one wants to be nitpicky, he was actually not part of the military,but the SS had military parts. What is much more sure: He was an absolute asshole.