I had the best time at Martin's Guesthouse. I was in a "princess" room, very charming and comfortable, with an amazing bathroom. Everything was very clean and well tended.
Most of all, Stephen was very welcoming and helpful,he gave me advice about sightseeing and the last day I could leave my bag at the guesthouse for the day until my flight.
Also, it's an easy walk to the old city centre (under 20 minutes), to the Meadows and Holyrood parks...