Klaus,
There are three possibilities: (1) bad DIMM(s), (2) incompatible DIMMs or (3) motherboard issue. Sounds like, if both DIMMs fail when running them individually, you've eliminated possibility #1. To check between #2 and #3, try testing with one of the DIMM types included in the compatibility list (see here: Intel Desktop Board DH87MC system memory)...
Hope this helps...
...Scott