It seems like I'm always playing catch up to my main competitors who have been around longer than I have been. I offer better services at better pricing but it seems like they have such a big overall lead on me in terms of brand awareness, more backlinks, bigger advertising budgets, better keyword rankings, etc. that I will never be able to make a dent into gaining more market share. How can I close the gap between my main competitors that seem to have gained a good market share over the years of being in business compared to my recent startup business?
Personally, I think you should be focusing heavily on your current users that are loyal to your brand currently and worry less on trying to buy users and get more brand awareness. To be frank you should be letting your strongest customers do all the promotion for you organically. "K-factor" This is how why you see businesses that get super big out of nowhere with practically nil paid marketing budgets.
One of the differences of the David v Goliath mentality is companies believe they need to take on Goliath. The truth is, however, your company can operate at a high level by offering your customers the best services, prices, quality, etc that you can and be focused completely on the customer experience.
The problem is by trying to compete with your competitors you are wasting all your money/time trying to beat or get market share vs. building a high quality business.
Focus on acquiring the customers you can through traditional paid, organic channels however stop focusing on trying to compete for market share. Focus on your product/business and ensuring that you have high retention %/ repeat users/payers, etc.
I worked at Lyft and to be honest we had that mentality vs. Uber who are essentially monsters compared to us. "Oh how will Lyft ever make a dent in Uber's market share" They have more money, resources, etc. The difference, however, was the customer experience with Lyft. Lyft, overall, is friendlier, and a better experience IMHO. Thus, you now see Lyft making major strides in market share in the ride-sharing economy.
At the end of the day, IMHO eventually the customer experience, product etc. will help you organically eat into enough market share.
Specific tactics depends on the business, but usually you should try to go for the niches for which your product is much better. There, are probably much more incline to switch (they usually feel neglected by the incumbent player)
The other thing that works well is optimizing what you do today in order to be more effective at acquiring customers and converting them - bigger efficiency can compensate to some extent for lower budget / resources.
The last group of actions is finding a way around the current information distribution system so you can bent the rules in your favour (this really depends on the nature of your business, segments targeted and the way the information flows). The usuall suspects are: getting refferal system for happy customers you can spread the world around (i.e Dropbox), affiliate programs, getting influential evangelist (i.e. Canva and Guy Kawasaki) or tones of small but with some coverage bloggers / local opinion leaders (i.e. Buffer)
Hope it helps
Disclaimer: This advice will only apply in certain cases.
Sometimes a domain upgrade will provide an instant credibility boost for your brand. A natural, authoritative domain name such as Apartments.com can reduce friction for word-of-mouth growth and also render SEM more cost-efficient.
That's one way to gain an advantage and level the playing field between yourself and more established players.
Maybe that isn't possible in your case. Impossible to say without knowing the circumstances.