Depending on the size of the chunks of concrete, you must expect long term settlement(years) when using "dirt". Sand will settle much quicker and give you a better idea on the amount of extra fill needed. You should always avoid "dirt", organics, silt and many clays if you expect a reasonable grade over a few months (with rain/snow) to get an idea of the amount needed to get reasonably level to invest in a lawn.
With the wrong fill, you could end up with a very long term maintenance problem to create a nice space with a level lawn.
Thanks Dick. This is what I was hinting at and I should have been more direct.