From what I was told years ago, Atherfield Holiday Center (or at least buildings on that site) were used to house Italian POW's, who had something to do with building the Milli Road. Brighstone Holiday Center housed Royal Marines and Fort Warden on t;other side of the island also housed Italian POW's.
The group of OAP's I had for a week on the coach were first time visitors to the island from Birmingham. Most were over 70 years old, many were on sticks & frames. When they booked the chalets they were told that Freshwater Bay was within easy walking distance and that the chalets were in a central location for all amenities - Newport buses etc - and that the center boasted a stunning and sfae beach that was easy to access.
Now anyone who knows Atherfield also knows that it is a very long way from Freshwater Bay...especially if not great on your pins as most of this group weren't.
I took them out for the day on the coach and they were literally begging me not to take them back to the chalets...they hated it.
So after making a few phone calls and a few personal visits I managed to get the group transferred to one of the larger hotels in Sandown for the remainder of their stay and they stayed there for a week every year from then on.
But the group were also told by the owner of Atherfield that the camp was used for POW's in both wars...and the group relayed that to me as 'nothing much had changed then'.