Deprivation of Liberty Safeguards (DoLS) should give people deemed not to have mental capacity the right to challenge their detention by the state.
But a Mental Health Alliance report will say the arrangements are bureaucratic and wasteful.
The government has said it will look carefully at the report's findings.
The Deprivation of Liberty Safeguards (DoLS) were brought into English and Welsh law in April 2009 and apply to people over the age of 18 who are in care homes or hospitals.
Those institutions have to apply for an independent assessment of anybody in their care who lacks mental capacity and is thought to be at risk of harm if they are not deprived of their liberty.
The safeguards include giving those detained, or their families, the right to challenge the restrictions, first locally by means of a review, and ultimately in the Court of Protection.
Robust deprivation of liberty safeguards are an essential protection for vulnerable people”End Quote Paul Burstow Care Minister But the report from the Mental Health Alliance, which represents a number of charities working in this field, will warn the scheme is "incredibly bureaucratic, wasteful, and the burdensome paperwork itself discourages many local authorities from using the legislation".
Speaking to BBC Radio 4's The Report, the author of the Mental Health Alliance study Roger Hargreaves warns of massive variation in the number of DoLS taken out by local councils across the country.
In the two years since they were introduced, West Sussex, for instance, made 206 authorisations whilst Hull made none.
Mr Hargreaves is calling for a radical rethink:
"If there are such variations, it raises questions about whether those safeguards are not being applied where the activity rates are very low, and alternatively [whether] they may be being overused in areas where they are very high."
'Complex issues'The damning report follows controversial cases like that of Steven Neary, a 21-year-old autistic man who was kept in a care unit for a year against his and his father's wishes.
Over the course of 2010, Hillingdon Council issued a number of standard authorisations for a Deprivation of Liberty on Steven Neary, even though, importantly, there was a less restrictive option for his care available at home.
The legislation is not meant to be used as a power by a local authority to detain someone against their wishes when other options are available.
Instead, it is meant to authorise restrictions in someone's care plan that are both necessary and in that person's best interests.Crucially, the legislation is meant to offer safeguards so that the person can challenge their situation.
In Steven Neary's case, a High Court judge ruled that Hillingdon had unlawfully deprived him of his liberty and had not followed the correct procedures.
In a statement, Hillingdon's director of social care, Linda Sanders, said: "Cases such as Steven's are hugely complex... As the judge said in his findings, at all times my staff were genuinely committed to ensuring that they did the right thing for Steven and had his best interests at heart.
"Before the court hearing, we recognised the need to improve our processes and had already made significant changes relevant to this case, including reviewing our training for those staff dealing with the complex issues relating to the Mental Capacity Act and Deprivation of Liberty Safeguards."
'Thoughtful analysis'While the Court of Protection is very effective in dealing with the cases it hears, the Mental Health Alliance suggests it is unsuitable as the avenue for appeal, as it is too slow, very expensive and hearings mainly take place in London.
Mr Justice Charles, a senior judge at the Court of Protection, said there were plans to have more hearings in district courts.
But he conceded that the legislation was hard to understand.
"It's complicated. What is deprivation of liberty? [If] you ask three people, you probably get four answers," he said.
Care Minister Paul Burstow told the BBC that he would look very carefully at the Mental Health Alliance report to see what lessons could be learned.
"Robust deprivation of liberty safeguards are an essential protection for vulnerable people. The Mental Health Alliance has offered us a thoughtful analysis of the problems some have encountered with the DoLS regime," he said.
The minister added that he looked forward to seeing the group's proposals to improve the system.
