Although none of us should be careless of the feelings of others, that does not mean we should not let them be responsible for their own feelings. People will inevitably become disappointed when the impose unfair, disrespectful, and sometimes manipulative expectations on others. Part of maturing is learning how to be considerate AND how to considerately set your own goals and courteously draw clear boundaries.
From what you have said here, it seems that your husband's family is nearby, but do not interact until the holidays. If you are not available at those times, there is a belief that they will be automatically offended. It seems to me that some effort could be made to give attention to nearby family at times other than the holidays. If you make the effort and they do not cooperate, then they really have no sound reason for you and your husband doing the fair thing and taking turns with family holidays.
There are more than 2 holidays a year. I have known families that would switch each year. One year they would have Easter and Thanksgiving with his family and the 4th of July and Christmas with her family. The next year they switched and everyone learned to be happy with that arrangement. Sometimes you just have to trust that people will adjust even if it is uncomfortable at first.
The sad reality is that our families do not take time to develop the art of consultation so that these types of decisions can be made and explained in a language that might avoid hurt feelings. As families grow and expand, these issues are always going to become a challenge and require adjustments. It should be expected and planned for.
One thing I learned to do as a mother is to think about the example I am setting for my child. Would I want my own son to feel manipulated over such things when he is grown and has a family of his own? Of course not! So, it is up to me to set the example of experimenting with ways to solve family dilemmas with dignity and loving care rather than with feelings of frustration, of being victimized, of assuming selfishness in others, etc.
I had a wonderful mother-in-law, but my own mother was prey to jealousies. It was easy for us to prefer to be at my mother-in-law's home for holidays. This was made a bit easier by the fact that my mother rarely offered to host and actually seemed relatively satisfied to share holidays with her children at their in-laws celebrations. But, there were many ways in which we had to divide ourselves in caring for parents. It helps if you explain your decision in terms of the principles involved. You can explain that the children need to have time with both sides of the family and that you want to be as loving, respectful, and fair as possible. It is important to remember that some people harbor immature feelings and may feel that your choice to be with the 'other' family automatically means you prefer 'them'. You cannot prevent people from thinking and feeling what they will. But, you can make sure you are not feeding such tendencies by taking the time to talk with them and express clearly that your motives are based in principle not in preferences. It isn't like we are in third grade and picking each other to be on our softball team. We are not 'voting each other off the island'. It is just so easy for people to feel rejected.
So, although I do feel we should care about the feelings of others and discuss these issues with them in a very caring way, we still have to grow up and know that we are all ultimately responsible for our own feelings and we cannot let the immature habits of others make our choices for us.
The real trick is for you and your husband to discuss these points and come to a unified understanding. Once the two of you have achieved a loving and respectful agreement, it probably won't be to hard to explain it to the rest of the family. One thing we learned to do as a couple was to ask ourselves if we were basing our decisions in love or in fear. That usually helped us to find agreements far more easily.
Make your holidays what you wish. It is perfectly legal!! Much as you might feel as if they are waiting to punish you for making your own decisions, there are no holiday police ready to toss you in jail.
Good Luck and HAPPY Holidays!