Good to hear from you. Thanks for the question! While I agree with you on eremitic life being a chance to live one's baptismal commitment radically, I think we disagree on this way of life not being a different way of doing so than for most people. We are not speaking about adding a bit more silence or solitude, or praying a bit more assiduously; all that can be a matter of temperament as well as commitment. Eremitical life, it seems to me, is truly different, more radical --- yes, but also different in its goals, methods, and purpose than most ways of living out one's baptismal commitments. On the other hand, I do agree eremitical life speaks to every Christian regarding the living out of baptismal commitments and can encourage them to imagine just what these commitments might really mean beyond some nominal, not to say lukewarm, version of Christian faith and spirituality. So perhaps we are in greater agreement on this than I actually know.
I don't think I have experienced people not trusting eremitical life once they understand it -- even just a little more than slightly. Mainly, I have felt that if they trust me they trust my vocation too, even when they are still seeking to understand it from the outside in. In other words, if the hermit is credible and genuine, then folks will tend to see the vocation in the same way. Usually what people push back against are the stereotypes that distort the vocation --- images of solitude that are pious versions of individualism and isolation and should be rejected, for instance, or indications of self-centeredness and selfishness that can be combated with a bit of personal attention to others and the needs of the community along with an emphasis on the centrality of love in such vocations. Most of the time, though, I think folks hold the idea of "hermit" at arm's length while thinking of me as a contemplative nun instead. That is the result of not understanding the vocation and not even knowing the questions to ask to begin to understand better. When I first began considering c 603 I was coming from a place that thought eremitical life had become obsolete. I had long begun to outgrow the old sense I had once had that contemplative life was a waste of skin (left over from my Protestant background I think but also buttressed by the importance of ministerial religious life). I suspect a lot of people still hold those views --- though they don't tend to express them to me. Again, my bottom-line experience is that if people trust me, they trust my vocation.
I will add to this that since I have begun working with occasional candidates for Canon 603 Profession, those few people who know about this work do ask more questions about eremitical life itself (we don't discuss the candidates, of course, except sometimes re what region of the country or world they come from). That there is a worldwide interest in the vocation and that it is slowly growing in the Church under c 603, does interest folks and they don't feel like they are asking too-personal questions at the same time. The pandemic also increased overt interest in eremitism and people asked me more questions (when I was actually around). Questions regarding healthy solitude, managing time alone, increasing or developing new approaches to prayer, learning to journal, etc are the kinds of questions I would get, especially at the beginning of the pandemic and correlative lockdowns.
I think there is a slim chance that folks who feel some call to eremitical life will push back against it. I guess I really believe the more prevalent reasons have to do with concerns regarding stereotypes, counterfeits, selfishness, and maybe not having enough sense of the vocation to even begin asking questions about it. I tend to find myself that candidates (or would-be candidates) for eremitical profession fall into three main categories, 1) serious candidates or inquirers --- those with a mature spirituality and the wisdom of experience that has led them to the desert, 2) romantics with naive senses of what the vocation is and entails including those looking for a sinecure, and 3) "nutcases" or eccentrics -- in this latter category I would include all kinds of people from those with significant mental illness to those who simply (or not so simply) want to use the vocation as a stopgap way to get professed despite being aware they do not have such a vocation.
In your regard (because you are a serious candidate for life as a hermit, and because I am sure people around you recognize that) what you may be experiencing is the tension between what people think they know about eremitical life (stereotypes, nutcases), and their own care and concern for you and your call by God to abundant life. It is unfortunate that many believe eremitical life to be narrow and cramped, with no room for creativity or realization of selfhood. Nothing could be further from the truth. Since, as you also mentioned, you are reading Cornelius Wencel's book The Eremitic Life you know, or will soon see, how strongly Wencel stresses the immense creativity of eremitic life along with its relationality and call to full personhood. Because of stereotypes and a strong sense of the inherent relationality of human beings, hermits often seem, at least initially, like they have embraced an anachronistic and anti-human life that could never be fulfilling or theologically justified. As the hermit grows in her vocation, these senses change for those who know her. I would bet some pushback comes from this fear for you among those who really care about or love you. Just saying!