There seem to be contradictory sections in the Basic Law.
Section 5 (b) The Prime Minister shall be a member of the Knesset. A Minister need not be a member of the Knesset.
This would seem to disqualify Olmert. However, if we look at sections 21(a) and 30(c) we get a different picture.
21. (a) Should the Prime Minister cease to function as a member of the Knesset, he will be deemed to have resigned; the Government shall be deemed to have resigned the same day.
Section 30(c) A Prime Minister who has resigned shall continue to carry out his functions pending the constitution of the new Government. If the Prime Minister has died, or is permanently incapacitated, from carrying out his duties, or if his tenure was ended because of an offense, the Government shall designate another of the Ministers who is a member of the Knesset and of the Prime Minister's faction to be Interim Prime Minister pending the constitution of the new Government.
What this seems to say is that the Prime Minister can continue carrying out his functions until a new government is formed even if he is not an MK.