Excellent and important questions. I have written about this before as well. Please see Hope, Shamelessly Persistent Trust. More recently, I am reminded of something Pope Francis said in his conversations with Rabbi Skorka in a section on prayer. Thus, in answering your questions I would like to take what Francis said a bit farther and perhaps also correct him a bit (I am not sure that I am actually doing the latter but I am sure I am doing the former.) What Francis said is this, [[[In prayer] there are moments of profound silence, adoration, waiting to see what will happen. In prayer there coexists this reverent silence together with a sort of haggling, like when Abraham negotiated with God for the punished citizens of Sodom and Gomorra. Moses also bargains when he pleads for his people. He hopes to convince the Lord not to punish his people. This attitude of courage goes along with humility and adoration, which are essential for prayer.]] Rabbi Skorka responds by saying in part, [[The worst thing that can happen in our relationship with G_d is not that we fight with him, but that we become indifferent.]] I think this observation too figures into my response to your question.
In the article I wrote about the dialogue Abraham has with God over the fate of Sodom and Gomorra I indicated that Abraham was the "Father of Faith" and that he personified the trust people of faith are supposed to have. I also noted that he personifies a journey the people of Israel themselves are called to make. It is a journey in which he and they come to know and trust the unfathomable depth of God's mercy. What I suggested was that it takes time to come to know God as one who acts according to a very different standard or notion of justice than the ones we ordinarily reason to ourselves. Over time Abraham and the People of Israel come more and more to know the God whose justice is his mercy, who sets things right in the world through his creative mercy and love, who is sovereign insofar as his mercy reigns, etc. Thus, story after story in the Old Testament recounts the faithlessness and sin of the People and the constant faithfulness, forgiveness, and mercy of God. What I want to call your attention to here is how, over time in continued encounters with God it is the people who change; they are brought to greater and greater faith but they are also brought to a sense of their own poverty, concupiscence, and recalcitrance when on their own.
I believe that we are charged with persistence in prayer not so we can change God's mind, but so that in that prayer and in our own encounters with God --- including his silence and refusal to give us what we think we want or believe is best for us --- we ourselves may be changed and our relationship with God may grow and mature. Persistence in prayer allows us to meet the God of Jesus Christ with our needs and desires as well as without attitudes of proprietariness, worthiness, competence, righteousness, selfishness, omniscience, etc, etc, and over time examine all of these in the face of a God who ONLY loves us and desires the very best for and from us. Most of our attitudes will change in such continued prayer; our perspective on any number of things will change: death, suffering, time, and so forth as God invites us to look beyond the immediate situation and find a greater hope and promise than we ourselves can even imagine. It is a bit like a person coming up again and again against that which is unchanging and, over time, changing themselves. In this case, however, they become more and more open to the actual answer for any prayer --- God's own presence and self --- and they come to know his faithfulness and presence no matter what else happens; their defining world becomes less merely that of time and space (though they will be made more capable of ministering within it) and more and more that of the Kingdom of God.
You see, in a real problem we especially don't want God as an added adversary or person we need to convince. We want him to journey with us and love and support us as only God can do, not be someone we are trying to convince and bargain with. Even so, in bargaining with the situation we will come, usually, to acceptance in ways we might not have otherwise. Bargaining is a part of grief, an arguing with our own pain and loss, a piece of coming to terms with a reality which has us helpless and powerless and articulating that bargaining to God so that he accompanies us in every moment and mood of our struggle is an entirely legitimate way to come to terms with reality. But only if we are persistent in it and use it as a way of working through our grief with God accompanying us AND, as you say in your question, not merely as a way of being stubborn or demanding God change things in the way we say God should.
The difference here is at least twofold: first any haggling is really one-sided --- we bargain (or better, we struggle) with our own pain and loss, our own lack of understanding and confusion and with inadequate notions of God. God listens and abides with us as our struggle with grief and loss challenge us to come to know ourselves and God better; God does not bargain. What looks like bargaining with God is really the outward appearance of our own struggle; God abides with us throughout and hears every movement of our hearts. It is not truly meant to change God because we know better, but rather, to come to know and share in his will to be with us in all things over time. Thus, it is marked by a corresponding openness to really hearing and accepting the will of God in whatever the situation is. In persistence we pour out our hearts to God and we do so again and again. In persistence we know that God is part of the answer, but we do not know precisely what shape that will ultimately take; as we continue to pray, we allow ourselves to become more and more determined to accept and even to aid God in that. Persistence is open to learning --- and to letting ourselves be shaped by the answers we will always receive. It is humble in its honesty, its openness, and in its naivete. Stubbornness, on the other hand, pretends to know what is best and how God should respond; it is closed to a deeper and higher wisdom, a more expansive vision of reality, or to the need to trust a God who is really mysterious in the best theological sense of that term.
While it is true God often does not answer our prayers with a "yes" in our precise terms, the problem with the "Sometimes God says no" answer is that it also presumes to know what God's answer is even as it does not allow us to continue importuning him. It short circuits the growth and maturation of the relationship that allows God to truly be God-with-us. It is an invitation to indifference and dismissal. The God who says no is not one we are usually open to walking with intimately on a daily basis or in difficult times. He is not one we can pour our hearts out to in all of our needs, weaknesses, distortions and darkness nor continue doing so until we ourselves eventually see the light or come to acceptance. Further, it continues to make of God someone who answers our prayers on our own level of understanding and expectation. It diminishes God and ourselves as well. My own experience is that God never says no. Whether we are at the beginning of a long "bargaining" process or months or years into it, God does not haggle; God always "says," "Here I am. Let me give you myself, my entire self, in this situation; let me live it with you. Let me transform you or situation with my compassionate presence. Let me deal with it and your own needs in ways you will one day realize are truly awesome. I promise, nothing whatsoever that you entrust to me will EVER be lost; all will be brought to life and completion in and with me!"