There are many kayak features to consider when choosing a kayak and all the options available are important. The following are some (but not all options) that might be important to you for recreational or touring kayaks.
First, you should decide what you want to use your kayak for - ponds, streams, lakes, rivers, ocean, etc. The right kayak for ponds and lakes might not be the right kayak for ocean (and it might be dangerous to consider using your kayak for the ocean based on the type of kayak you decide upon).
How heavy and how high can you and another person be comfortable to lift?
Can you lift the kayak by yourself? Do you always need two people to load the kayak on the vehicle or into the storage area?
Where is the kayak going to be stored?
Where are you going to store your kayak? What is the largest size kayak that will fit into the storage area? Is there a size restriction in your storage area? Do you need to consider another place to store your kayak?
Are you going to use it for recreation, light touring or touring?
Recreation kayaks are usually a small kayak, more stable, easy to pack and get away for the day, day paddling kayak, easier to turn in a smaller radius and good for quiet waters.
Light touring kayaks is farther paddling, paddling a longer distance than the recreational kayak, maybe a day overnight, has at least 2 hatches, heavier than the recreational kayak and probably needs 2 people to load the kayak on top of the car.
Touring kayaks is paddling the farthest, have at least 2 hatches, is longer than the light touring, has extra safety features (tow lines, etc.) and is much safer on the sea.
Shapes & sizes?
There are many shapes to consider (too many to even mention here). If you sit in the kayak on the floor at a kayak retail shop and the kayak is flat on the floor, it will be a more stable kayak in the water. The kayak will be harder to paddle, harder to keep on a straight track and harder to turn. If you sit in the kayak on the floor and it is harder to keep upright, the kayak will paddle faster and will be easier to turn. The rule is the safer you feel initially in your kayak, the faster you will outgrow your kayak. Wide kayaks are 28" or higher wide and narrow kayaks are 19"- 27". The narrow kayaks will paddle faster, but will feel very tipsy until you become used to the kayak.
Skeg, rudder or nothing?
The skeg and the rudder will keep the kayak going straight under troubling conditions such as high wind or strong currents. Contrary to most thinking, the rudder is not to steer a kayak. But the rudder will get in the way during rescues and is just another part of the kayak to break or need repair. A stone might easily become caught between the skeg and the wall of the kayak. This may not allow you to lower the skeg during inclement weather conditions. If there is no skeg or rudder on the kayak, you have to learn and practice your kayaking skills to keep the kayak going straight. In very bad conditions, the rudder is an option that makes paddling much easier and it might make a difference of not being in trouble and being in trouble.
Fibreglass, Kevlar, Molded Plastic, High Impact Plastic, Wood and more?
What material is best suited for the type of paddling you will be doing? Where will you be paddling and what will be the conditions of the water and the land? Will the land usually be rocky, have wood stumps, sand or dirt landing areas? Molded Plastics are more durable, can be pulled up on the rocks and enjoy very rough treatment. These kayaks are usually the heaviest of the kayaks. Fibreglass is much stronger, needs to be treated very carefully, weighs less than Molded Plastics and usually more than the High Impact Plastic kayaks. You have to be careful not to hit rocks, not to land on rocky land, not to drop the kayak, etc. Fiberglass repairs might have to be made on a fibreglass kayak. High Impact Plastics give you the benefit of strength, easier to maintain than Fibreglass and are usually lighter than Fibreglass. Kevlar is usually the lightest, the strongest and the most expensive kayak. The kevlar kayak is the strongest kayak, but the hardest to repair. Hairline cosmetic cracks usually appear after many years of use and are difficult to repair. The kayak prices go from the most inexpensive to expensive: Plastic, High Impact Plastic, Fibreglass and Kevlar.
Hatches or no hatches?
Hatches are a very important feature of a kayak. The hatches keep the kayak floating when the kayak is not upright and also limit the amount of water that can fill into the cockpit. They also protect everything from becoming wet that is carried inside the hatches. If the kayak is an open kayak (no hatches), special bags HAVE to be purchases to fill in the bow and stern to keep the kayak from sinking.
Knee braces or no knee braces?
Do you expect to paddle in the ocean or in calm waters. If you expect to paddle in the ocean or in any body of water that has waves, strong currents, etc., the kayak you choose should have knee braces to help you steer and paddle. Do the knee braces fit you and are they comfortable?
What color do you want?
Kayaks come in a multitude of colors and some colors fade or scratch easier than other colors. When you select colors, ask the manufacturer if those colors will hold up well under the sun, salt water and all the elements.
Deck lines, compass, tow line, bungee cords, etc.?
Depending of the type of kayaking you expect to do, all the extras on the kayak make paddling less dangerous. If you will be paddling in the ocean, choose a kayak that has the most features available. Learn what each feature does and is that feature important to you?
How does the kayak fit and feel?
Never choose a kayak unless you have paddled in the water. You will be disappointed and even may want to return the kayak if you haven't paddled in the kayak. A kayak is similar to shoes. You would never buy shoes without trying them on. Just because your friend says that a certain model is the best of the kayaks, it doesn't mean that the kayak will fit you. Everyone is different and every kayak fits everyone differently.
Don't forget to learn how to maintain your kayak correctly because incorrect methods might damage your kayak and choose a kayak wisely.