Oliver Letwin’s recent Bright Blue lecture on the theme of ‘Opportunity for All’ presented a solid case that the current government’s policies were – slowly but surely – improving life chances for people right across the socio-economic spectrum. He was at pains to make clear, however, that this was not the limit of his ambition – he wanted to achieve “real equality of opportunity for all”. While he acknowledged that this was a high bar, he felt it was one that could, and should, be cleared. He scorned as “cynics” those who maintained it could not.

It is with no pleasure that I find myself in the “cynics” camp. Letwin identified education as the most important factor in providing equality of opportunity and praised the changes that have been made in our schools system in recent years. Afterwards, a questioner from the floor accused Letwin of neglecting the importance of early-years initiatives – a view he rebutted convincingly enough. But Letwin's argument also left him open to the interpretation that his conception of equality of opportunity ends at 18, or at least after further education. It did very little to address the ongoing impact that existing patterns of wealth – and in particular property ownership – can have on true equality of opportunity.

Imagine the case of two young people, similar in every respect except that one comes from a markedly more affluent background than the other. Let us assume, for the sake of argument, that educational inequalities have indeed been overcome, at least in this case. Both our young people study hard, perform strongly at school and gain a place at the same good university. They graduate at the same time and with the same degree. Because there is also equal opportunity in hiring practices, they both get jobs as graduate trainees at the same firm and on an identical salary. So far, so good.

Now let’s start to factor in the differences between them.

One has had their university fees and living expenses met in large part by their parents, who have also generously put a substantial sum of money towards the deposit on a flat near the centre of the town where both our new graduates now work.

The other – even if they have worked during term-time and vacations – will probably have several thousand pounds’ worth of student debt which, even if not immediately payable, will eventually make a fair-sized dent on their disposable income. Unable to afford to buy or even rent a town-centre property, they will need to live further out. Even allowing for this, given the constraints on housing supply, they may end up paying more in rent than their counterpart does to pay the outstanding mortgage.

Living further out imposes costs of its own. Commuting is more expensive and time-consuming.  Socialising and networking out of hours – vital to success in many careers – makes a greater demand on someone with an hour or more’s journey home afterwards, than on someone who can just jump on a bus or into a cheap minicab, or even just walk.

Just as importantly, the very fact of an affluent background provides a greater sense of financial security. That’s not just about help with buying a home, hugely important as that undoubtedly is. It’s also about the knowledge – less tangible but very real – that there’s something to fall back on if things go wrong and that, at some point hopefully long-distant, a sizeable inheritance may be in the offing, a financial silver lining to an unwanted cloud on the horizon.

How much easier, with this comfort blanket draped over your shoulders, to take the sort of risks that are sometimes needed to take your career to the next level? To take a different line from your bosses and stick to your guns, in full knowledge that this could end with your being fired just as easily as being proved right. To ask for a pay rise or a promotion when you think you’ve deserved it, and be prepared to quit if you don't get it. To start up on your own to take things in the direction you want, or if you think you have seen a gap in the market.

None of this means that a young person from a less affluent background cannot succeed. Nor does it mean that a wealthy family is any sort of guarantee. But it does suggest that – sadly – "real equality of opportunity for all" may be more difficult to achieve than Letwin supposes.

Mark Stockwell is a former adviser to the Conservative party. He is a member of Bright Blue and now works in public affairs.