He met Congress president Sonia Gandhi and submitted his resignation letter in line with the one person, one post principle adopted by the party at the Udaipur Chintan Shivir.
Sonia, the chairperson of the Congress parliamentary party, will now appoint a new leader and convey the decision to the Rajya Sabha Chairman.
The question now is who will succeed Kharge as the Leader of Opposition in the Rajya Sabha. While sources said the party is no hurry to take a decision, the buzz is the post could go to a leader primarily from the Hindi heartland.
Sources said Digvijaya Singh, who bowed out of the presidential race, Pramod Tiwari and also Mukul Wasnik, who is from Maharashtra, could be in contention.
With Kharge, who is from south India, looking set to become the party president and the party already has Adhir Ranjan Chowdhury as its leader in Lok Sabha, it could be looking at a person from the Hindi belt as the Leader of Opposition in the Upper House to balance regional equations.
If the leadership decides to look beyond north India, former Union minister P Chidambaram could be considered.