PM responsible for Pakistan defeat