Perhaps the simplest and easiest to understand is the argument of the First Cause. (It is maintained that everything we see in this world has a cause, and as you go back in the chain of causes further and further you must come to a First Cause, and to that First Cause you give the name of God.) That argument, I suppose, does not carry very much weight nowadays, because, in the first place, cause is not quite what it used to be. The philosophers and the men of science have got going on cause, and it has not anything like the vitality it used to have; but, apart from that, you can see that the argument that there must be a First Cause is one that cannot have any validity. I may say that when I was a young man and was debating these questions very seriously in my mind, I for a long time accepted the argument of the First Cause, until one day, at the age of eighteen, I read John Stuart Mill's Autobiography, and I there found this sentence: "My father taught me that the question 'Who made me?' cannot be answered, since it immediately suggests the further question `Who made god?'" That very simple sentence showed me, as I still think, the fallacy in the argument of the First Cause. If everything must have a cause, then God must have a cause. If there can be anything without a cause, it may just as well be the world as God, so that there cannot be any validity in that argument. It is exactly of the same nature as the Hindu's view, that the world rested upon an elephant and the elephant rested upon a tortoise; and when they said, "How about the tortoise?" the Indian said, "Suppose we change the subject." The argument is really no better than that. There is no reason why the world could not have come into being without a cause; nor, on the other hand, is there any reason why it should not have always existed. There is no reason to suppose that the world had a beginning at all. The idea that things must have a beginning is really due to the poverty of our imagination. Therefore, perhaps, I need not waste any more time upon the argument about the First Cause.
Clayton says: I don't see that Russell failed to take account of this point: If everything must have a cause, then God must have a cause. If there can be anything without a cause, it may just as well be the world as God, so that there cannot be any validity in that argument.You say, "Cosmological arguments always tell you what needs a cause. Contingent things. Things that begin to exist". He doesn't think that cosmological arguments show both that all contingent things need causes _and_ the world is such a contingent thing. He doesn't think that cosmological arguments show that everything that has a beginning needs a cause _and_ the world had a beginning. Now, you might disagree with _this_ point, but I don't think Russell is guilty of quite the strawman you've suggested.
But it seems to me that there are attempts on the table (Russell could have been forgiven for not knowing about the Kalam argument, but the Thomistic argument is another matter) that try to point to a characteristic that the physical world possesses, namely contingency, which God does not possess, such that the world needs a cause and God does not. These attempts may fail, but Russell surely knew that they existed, and nevertheless he presents a one-parapraph refutation of all cosmological arguments that simply presumes that all attempts like this fail. In the process he makes theists look really retarded, because it looks as if advocates of these arguments simply had to be reminded of the simple point that James Mill made to his son John Stuart Mill, and the cosmological argument is a cooked goose. In fact a good deal of the impact of the paragraph has to do with not only that the argument can be refuted, but that this is something that can be done on one's lunch break.
I suppose you can say that here Russell is giving us the "short version" of an argument that can be defended at greater length. And of course lots of people do that sort of thing. You might think that in fact the universe has no cause-requiring properties that God would not equally possess. But in any event he makes it look easy, when it really isn't.