Gordon Strachan's Middlesbrough tenure got off to a bad start as Jamie Mackie scored the only goal of the game for strugglers Plymouth.
Boro fell behind 19 minutes into the second half, when Mackie slotted past Brad Jones from 12 yards.
The hosts should have levelled when Kari Arnason fouled Sean St Ledger in the box, but Adam Johnson drilled the resulting penalty wide.
Argyle held on for three points despite seven minutes of stoppage time.
Home fans at the Riverside Stadium endured another dismal performance in Strachan's first game in charge, and victory was no less than Plymouth deserved following a spirited display which belied their lowly league position.
Middlesbrough manager Gordon Strachan:
"It was frustrating. It always is when you lose. But it was nice to be back. I enjoyed it, not the game but the experience.
"It will take hard work for us to find a winning formula. In terms of creating chances and getting the crosses in we don't have a problem but that formula in terms of scoring goals is just not there. Maybe a bit of that is in the head.
"You can only do it by hard work in training and hard work on the pitch.
"We need to finish the chances but then we all know what's wrong. You don't need to be a Uefa qualified pro to see that. We'll put it right."
Plymouth manager Paul Sturrock told BBC Radio Devon:
"There has been a lot of work put in over the last week, I thought the shape and organisation of the team was different class.
"They showed again today that they are willing to work hard as a unit.
"I have always known Yoann (Folly) has had the ability and he showed today how influential he can be."
www.bbc.co.uk
