This is an easy one for me to answer, since a quick look at the big Oxford English Dictionary gives the historical details. The fruit definitely came first—it is recorded in English in the fourteenth century, while the application of its name to the colour only appeared at the beginning of the seventeenth. This raises the question of what people called the colour before they had a word for it: either they didn’t (few things in nature are that colour and there was no bright orange pigment available to artists and dyers until the early nineteenth century) or they borrowed terms like yellow, gold, amber, or red to describe various shades.
By the way, the word orange is interesting, etymologically speaking, because it’s an excellent example of a change called metanalysis in which the first letter of a word shifts to the end of the preceding word. So a numpire became an umpire, a napron became an apron, and so on. In Arabic, the fruit was named naranj (from Persian narang and Sanskrit naranga—the orange may have originated in northern India) and this name came with the fruit into Italian and also into Spanish, in which the fruit is still called naranja. The initial letter dropped off before the word reached English, possibly in Italian but more probably in French.
