Having dirt bike experience would make all the difference. Graded dirt/gravel roads are considered “easy” by dirt riders, but you need to know what you’re doing, how to position your body/bike and manage traction in turns to not wash out. It’s pretty different from pavement riding.
I took my GS500 on a dirt/gravel road, hilly and twisty, about 10 miles long, that would have been easy for a dual-sport or dirt rider, but difficult for a sportbike rider. I rode mountain bike a lot and love dirt, but on a motorcycle (way heavier) with street tires it’s a whole different ball game. I was practically crawling in the beginning, going 15-25 mph, where a real dirt rider could easily go twice that speed. I was a little scared the whole time, but I liked that. My biggest worry was the GS not having enough suspension to deal with the big bumps, but that wasn’t too bad on a graded road.
I rode the same road on my DRZ400sm, thinking it would be easier, but not really because that time they laid down some deep gravels in some sections of the road. Again, it’s a matter of learning how to ride dirt.