Hey mudchet, you should come back sometime, I've been making maps, sifting through data, got a list of access roads I haven't driven that look like they might be worth exploring, and if those don't pan out, I know several more that are definitely 4x4 required.
I've had my X in Turkey Bay, like the others said, the main trails are not that bad, you get off them and on harder trails, you can get in over your head quick. I wouldn't bother with it, I don't really plan on taking my X back in there, way to much other ground to explore first.
I highly recommend the access roads, they can actually be significantly more technically challenging than Turkey Bay in places, and more fun for a Xterra sized vehicle. Traffic is better as well, most of the time I explore a trail it looks like I'm the only one that has been there in a while. On the more popular trails you might see a few other 4x4's, especially on a weekend.
Don't go alone in a 2WD unless you are prepared to turn back at the slightest challenge. I've been stuck out there in a vehicle that wasn't up to muster, and it had AWD. Not a good feeling and a lesson learned. It's all good until you find out that mud hole was a bit stickier than you assumed, and then your working hard to get back out. Cell coverage is spotty, and it's not like it's a highly populated area, you will be out there a long time if you cannot self recover. But don't let that scare you, it's just cautionary and I have no idea of your experience of capabilities so I throw it out there. LBL is the perfect place to take a 2WD high clearance vehicle and have a lot of unpaved roads to explore.
The access roads are generally under rated, most people don't know where to start, the more obvious roads closer to the main attractions get maintained better, hence you can quickly end up only finding 2WD friendly gravel roads on your initial unguided explorations. You have to study the maps to know what is legal to drive on, but there are still many miles of barely maintained dirt to see. FREE and completely legal. But if you know where to look, you can find some that are challenging. Maybe not hard core for built rigs, but for stock to mildly modified Xterra's it's about right. Some of the mud holes out there can be scary deep, they are few and far between but they are out there, I get out to make sure I'm not going to dive into a waist deep hole before I go into anything big enough to hide a hole like that.