Vendor assessments are usually done once, thoroughly, and then never again, because nothing about the first one made the second one cheaper. The fix is unglamorous: fix the criteria, fix the weights, score the same way every time and record why you concluded what you did. A comparable score is worth more than a precise one, and a recorded reason is worth more than either.
Compare on the same weights, or you are not comparing
Two suppliers assessed against different criteria have not been compared, they have been described. Fix the criteria and fix the weights before scoring anything: delivery, quality, price, responsiveness and compliance, weighted once. On the worked example weights of 30, 30, 20, 10 and 10 against scores of 8, 7, 6, 9 and 8 give 74 out of 100.
Sort the list by category, not by name
Duplication is invisible in an alphabetical list of company names and obvious the moment the same list is sorted by what you buy. Two suppliers in one category is either a deliberate second source or an accident, and until the list carries a category field nobody can tell which. This is the field that pays for itself fastest.
Consolidate where switching is cheap, not where spend is high
The instinct is to consolidate the biggest lines, which are usually the hardest to move and the most carefully priced already. The easier wins are in categories with several small suppliers, duplicated software, and anything where switching costs a phone call. Single-sourced suppliers are the ones to leave alone, or to deliberately second-source.
Write down why, because the question returns
Whatever you decide, record it against the vendor: the score, the alternative you looked at, the price movement, the reason. Next year the same question arrives and the person answering it may not be you. An assessment with no recorded reasoning has to be redone from scratch, which is why most businesses only ever do it once.
Questions people ask about vendor assessments
How often should suppliers be assessed?
Tie it to the contract rather than the calendar: before the notice window opens on anything significant, and when something changes on everything else. Annual review as a standing plan is the stage that quietly never happens.
How do we compare two suppliers fairly?
Same five criteria, same weights, same scorer, and show each criterion's contribution rather than only the total. Fairness here means comparability, not objectivity, which is not available.
Where is consolidation actually worth doing?
Categories with several small suppliers, duplicated software, and anything where switching costs a phone call. Not the biggest lines, which are usually already sharply priced and expensive to move.