The 1824 painting, entitled Beaching A Boat, Brighton, is understood to have been looted and smuggled out of war-time Hungary.
A committee of government-appointed experts said the Tate had "a moral obligation" to return the painting.
The artwork became part of the Tate's collection in 1986.
It was donated to the Tate by Mrs PM Rainsford.
The picture, which is believed to have been stolen in 1945, was first recorded in London in 1962.
It will be returned to the family of the original owner, who died in 1958.
The report said the unnamed man was "a well-known Hungarian artist and connoisseur from a Jewish background" and described how he fled the 1944 Nazi invasion and "went into hiding".
The Spoliation Advisory Panel said: "His properties were confiscated, and contemporary witness accounts noted German military trucks being loaded with effects from the castle and being driven away."
The claimants discovered two years ago that the painting by Constable had ended up in Tate's collection. The claim was formally submitted to the spoliation panel in April 2013.
The panel was critical of the Tate for failing to thoroughly investigate the painting's history, given "the possibility that the painting had been the object of spoliation".
"It would not have been difficult to have made enquires of the Hungarian government, [which] included the painting on its official  list of looted art from the late 1940s," the report said.
It also suggested the gallery had not furnished the heirs with all the relevant information about the artwork.
The Tate said it was "grateful for the care with which the Panel has examined the evidence and is pleased to follow the conclusions of the Report."
It added: "Tate will therefore recommend to its Trustees, when they next meet in May, that the work be returned to the claimants."