My husband and I went over Memorial Day last year and didn't find it crowded at all. We just got back last week and it was less crowded but not by much. It seemed to be 'business as usual' over the holiday weekend last year with no closed shops.
We've gone in May several times and our last May trip overlapped Memorial Day. It seemed pretty normal to me. I think May has gotten a little more popular lately though judging from how many people have been on island recently.
We live in Point Pleasant Beach! Escaping the crowds that will come down this w/e to start off their summer!!! Already went to our first Beach Bar grand season opening on Saturday! Had a blast with a live band, free buffet and flowing drinks! Even had a Mojito to start off my vacation 1 week early!!!