I once had to do something similar, and I've used two different solutions.
1) Simply add an alpha channel to the image, or create a separate grayscale alpha map to use alongside it.
2) Inset the mesh, with the resulting face being close to the edges (or further, as desired). Then go to Vertex Paint mode, and set the vertices along the edge to black, and the other vertices to white. The Vertex Colors are exposed to cycles via the Input->Attribute node, where you type in the name of the Vertex Color dataset (Col by default).
To get the transparency, you plug the alpha (either a map, channel, or vertex color attribute) into the factor input of a Mix Shader node, with one input being Diffuse BSDF (or whatever you're using) and the other being a Transparent BSDF node. You can use the Color->Contrast node on the alpha input to make it less gradual, or use a Converter->Color Ramp instead.
The image below demonstrates the second technique, using vertex colors: